On 4 February 1759, Oliver was married to Isabella Sarah Newman (d. 1777), a daughter of Richard Newman. Together, they lived at
Castle Oliver in County Limerick, Ireland. They were the parents of: •
Charles Silver Oliver (1763–1817), who married Maria Elizabeth Morris, daughter of banker
Abraham Morris, in 1805. son of
John Thomas Waller. • Elizabeth Oliver (b. ), who died unmarried. • Robert Oliver (–1834), who married Sarah Shepley (née Barker) Sotheran, widow of William Sotheran and daughter of Edmund Barker, in 1809. Oliver died on 21 November 1798. ==References==