Elias was born in upstate
New York. He inherited his left-wing politics from his parents. His father was a doctor, the son of immigrants to the US from Hungary who spent some time in
Spain in the 1930s during the
Spanish Civil War. His mother was a librarian and trade union activist, whose parents came to the US from
Russia. The family moved to
Woodstock in the early 1960s. Elias attended Woodbourne Elementary School, Fallsburg High Central Junior and Senior High School. From there he went to
St. John's College in Annapolis, Maryland, graduating in 1962 with a degree in mathematics and philosophy. He was President of The Rogers Albritton Film Club and acted in The King William Players productions of
Shakespeare,
Brecht, and
Beckett. Elias is the brother of
Ruth Rogers (Lady Rogers) who is married to British born architect
Richard Rogers (Lord Rogers) and Susan Elias, an artist living in Berlin who is married to artist
Reinhard Voigt. Elias has been married three times: Caroline Bielefelt 1963–1974, Laraine Mestman 1991–2001, and Bianca Roberts former executive director of the
Mona Bismarck American Center in Paris. Elias has one son with Laraine Mestman, Frederick Mestman, and three step-sons: Ernie Klein, Siggy Bodolai, and James Bulliard. ==Acting career==