Pasaribu was married to Isna Hirawati and has two sons and one daughter. One of his sons, Dolly Pasaribu elected as Regent of
South Tapanuli for the period 2021–2026. His older brother was
Minister of Manpower in 1999–2000. His younger brother was also involved in politics, namely who was the Regent of
South Tapanuli (2010–2021), and who was a member of the
Indonesian Parliament since 2014. ==Death==