Baker was twice married. His first wife Elizabeth was buried on St Kitts in 1745. There was a son of this marriage, John Proculus Baker. He married in 1773 Ann Susanna Pool, daughter of the Rev. John Pool. John Pool(e), aged 61 in 1774, acted as attorney in Jamaica for absentee planted, and had reputedly made a fortune. Baker married, secondly, Mary Ryan (died 1774), elder daughter of Thomas Ryan (died 1755) of Montserrat, with whom he had a family of five sons and two daughters. Mary was Catholic, and the elder daughter, Martha (1747–1809), was given a Catholic upbringing and education definitively from 1763. It proceeded after the end of the
Seven Years' War at a convent of the
Ursulines in
Lille, and then shortly afterwards at another Ursuline convent, on
Rue Saint-Jacques, Paris. Martha Baker met in Paris
Henry Swinburne, of an old
recusant family, who was taking a version of the
Grand Tour. They were married in 1767, in
Aix-la-Chapelle. They had a family of four sons and six daughters. Martha shared after her father's death in 1779 in his West Indian property, but the
Anglo-French War by then being fought in the Caribbean made it worthless. Henry Swinburne pursued legal action and used influence with
Marie-Antoinette to try to redress the position. Martha at her death had a share in an estate on
St Vincent.
Christopher Hewetson, the Irish sculptor in Rome, made a bronze bust of Martha Swinburne c.1778. Philip Chesney Yorke, who edited Baker's diary, was a descendant. He was a grandson of Rear-Admiral Reginald Yorke, and his wife Harriet Walker, daughter of John Walker of Purbrooke Park. Harriet Walker's mother was Maria Theresa Henrietta (Harriet) Swinburne, daughter of Henry and Martha Swinburne. ==Notes==