Previous to his marriage, he had a relationship with the French
courtesan Pauline 'Cherie' Pacquot. The affair produced one son, William Henry Browne (1813-1817), whose paternity Howe questioned upon discovering he was not Pauline's only lover. He paid £1,000 a year to support Pacquot and his son. On 4 March 1816, Browne married the 16-year-old Hester Catherine, daughter of
John de Burgh, 13th Earl of Clanricarde. His mother is said to have remarked that "the most remarkable likeness to Pauline that I ever saw". Their first child, Louisa Catherine was born at Westport House in December 1816. Their second daughter Elizabeth was born in December 1818. In January 1820 his oldest son
George was born and named in honour of his godfather, King George IV. A second son Howe who was born in January 1821 died in Florence in February 1822. A third daughter Catherine was born in April 1822. James de Burgh (‘Jem’) was born in 1823.
John was born in Westport House in 1824. Harriet was born in Paris in 1827. Emily Charlotte was born in Westport House in 1829. A son
Henry Ulick was born in March 1831. A son Richard was born in Jamaica in August 1834. Hester Georgina was born in July 1837 and Augusta in August 1838. Their last child Marianne was born in 1839. ==Governor of Jamaica==