On 30 June 1915, he married the Hon. Alathea Margaret Gwendolin Valentine Gardner (b. 1893), the second daughter of
Lord Burghclere. Her paternal grandfather was
Alan Gardner, 3rd Baron Gardner and her maternal grandfather was
Henry Herbert, 4th Earl of Carnarvon. Alathea's younger sisters were Mary Gardner, who married
Geoffrey Hope-Morley, 2nd Baron Hollenden, and
Evelyn Gardner, who was the first wife of author
Evelyn Waugh. Together, Sir Geoffrey and Alathea were the parents of one child: •
Ann Jennifer Evelyn Elizabeth Fry (1916–2003), who married
Robert Heber-Percy in 1942. They were the parents of Victoria Gala Heber-Percy (wife of engineer and inventor
Peter Zinovieff) before they divorced in 1947 and she then married the poet
Alan Ross. In 1921, he bought
Oare House in Wiltshire, which was his home for the next forty years (and was later owned by
Sir Henry Keswick). Fry hired architect
Clough Williams-Ellis to remodel the 1740 home in the early 1920s. Sir Geoffrey died in 1960. ==Arms==