His country residence was at
Cheam,
Surrey. In 1797 Smith married Caroline, daughter of
Richard Vernon, M.P. for Tavistock. She was half-sister of the mothers of the third Lord Holland and of the third Lord Lansdowne. By her Smith was father of
Robert Vernon, who became a prominent
Liberal politician and was created
Baron Lyveden in 1859. Smith died on 10 March 1845 at his house in
Savile Row, London. ==References==