Although van der Smissen was never married, he had a long-time relationship with one Henriette Dubois (1825–1895), and they had a son, Alfred, born in
Lille on 21 July 1855. Despite this, the elder Alfred never married Henriette. In fact, their relationship was eventually characterized by significant friction; in his final testament, van der Smissen referred to his son as "the grief of my life" and ultimately disinherited him in favor of his younger brother, Adolphe. The younger Alfred survived his father by six years, dying in Paris on 15 May 1901. It is also widely believed by historians that van der Smissen had an affair with Empress
Charlotte of Mexico and that the Belgian-born French general
Maxime Weygand was the result of the affair, citing Weygand's strong physical resemblance to him. ==Honours and arms==