He was married twice, first to Violet Houldsworth (d. 1927) and subsequently to Ellis Johnsen (d. 2001). He had a son and a daughter by his second wife. He was a keen traveller, including a visit to the
Empire State Building in New York in 1931 when it opened. Photograph albums and scrapbooks from his travels are in the Essex Record Office. Orsett church contains
hatchments to his father and his first wife – the latter was painted by Whitmore. The other hatchments in the church were restored at the expense of Whitmore following a fire. He also painted portraits, two of which are in the Thurrock Museum. Whitmore died in 1962 and was buried with full military honours at Orsett parish church. The funeral was an important local event and the head gardener on the Orsett Estate, Alfred George Cuthbert, took a number of photos of the funeral for his album. Four of these were published in 2018. The baronetcy and estate were inherited by his son, Sir
John Whitmore (racing driver). He is commemorated in the name of a local pub, The Whitmore Arms. His portrait from W.W.1 hangs in the headquarters of the
70th (Essex Yeomanry) Signal Squadron. A later portrait in Lord Lieutenant's uniform by
Herbert James Gunn hangs in the Shire Hall in Chelmsford. On 10 September 2015, a Thurrock Heritage plaque was unveiled commemorating Whitmore and other Officers and Men of the Essex Yeomanry and 10th Royal Hussars who served in the Great War. ==Notes==