The only child of wealthy banker Thomas Whyte, Esq. of
Kirkcaldy,
Martha Whyte married
Charles Bruce, Earl of Elgin and Kincardine, at
Edinburgh on 1 June 1759. Their eight children were
Thomas Bruce, 7th Earl of Elgin (1766–1841), known for the removal of the
Elgin Marbles from the
Parthenon in
Athens; Lady Martha Bruce (1760–1767); Lady Janet Bruce (1761–1767); William Bruce, Lord Bruce (15 January 1763 – 27 March 1763);
William Bruce, 6th Earl of Elgin (1764–1771);
Charles Andrew Bruce (1768–1810), Governor of
Prince of Wales's Island;
James Bruce (1769–1798), Member of Parliament for Marlborough; and Lady Charlotte Bruce (28 May 1771 – March 1816), who married Captain (later Admiral)
Philip Charles Durham. In 1762, Lady Elgin's portrait was painted by the fashionable
Allan Ramsay, who the same year painted
King George III. Lady Elgin's eldest son, William Robert, born in 1763, lived only ten weeks. Her two eldest daughters, Martha and Janet, died in 1767 at the ages of seven and six. On 14 May 1771, she was widowed, and only two months later her second son, also called William Robert, the new Lord Elgin, died at the age of seven. Her remaining four children, three sons and a daughter, all lived to adulthood, but on 10 July 1798 her youngest surviving son,
James, then aged twenty-nine, was drowned while crossing the
River Don at
Barnby Dun in
Yorkshire when his horse was swept away by the stream. ==Later life==