James Goldsmith nació el 26 de febrero de 1933 en París, Francia. Estuvo casado con Annabel Birley, Ginette Lery y Maria Isabel Patiño. Murió el 18 de junio de 1997 en España.
Fathered Zac Goldsmith and Jemima Khan while their mother (the sister of Jane Rayne) was married to her first husband, and he was married to his second wife; they married in 1978. Goldsmith had fathered a son by his second wife before marrying her, then fathered two children by a mistress while married to his third wife.
[on the staff of "Private Eye", during his lawsuit against its editor}: They have attacked my son. I will throw them into prison. I will hound their wives, even in their widows' weeds.