Dickerson was born July 25, 1933, in
Topeka, Kansas, to George Graf Dickerson, a lawyer, and Elizabeth Dickerson (née Naumann); he did not develop a good relationship with his parents. He had one brother, five years his junior. As a child, his family lived in
Michigan, the South Side of
Chicago,
Queens, New York, and
Virginia. From 1965 onwards he lived in the same apartment in
Manhattan once rented by critic
James Agee (with whom Dickerson claimed to have spiritual contact). Dickerson served in the U.S. Army from December 1953 to the fall of 1954. Dickerson married four times and had five children: two daughters by his first wife, a son by his third wife, a daughter by his fourth wife, and a son born out of wedlock with a Finnish journalist. He was romantically involved with 1960s supermodel
Veruschka. ==Writing==