On 2 September 1837, at
St Michael's Church, Highgate, Chester married firstly Anna Maria Isherwood, a daughter of Robert Isherwood, Esq. They had a son, Robert, and two daughters, Dulcibella Fanny and Caroline Mary. His wife died in 1854 in the parish of
St George Hanover Square. On 24 March 1856, Chester married secondly Henrietta Mary, daughter of George Goff. Probate on his estate was granted to his widow, Henrietta Mary Chester, of that address. In 1871, Mrs Chester was living in
Brighton with her children Ella, Leonora, Henrietta, and Harry Chester, her step-daughter Caroline, and eight servants. The
census of 1901 finds her at Poplar Hall,
Birchanger,
Essex, with a small grandson and five servants, and gives her birthplace as
Florence. ==Published work==