Gray was born in New York City on July 14, 1837. He was a son of Susan Maria (
née Zabriskie) Gray (1814–1904) and John Alexander Clinton Gray (1815–1898), a dry goods merchant in New York City. His older brother was Albert Zabriskie Gray, warden of
Racine College, and his younger brother was jurist
John Clinton Gray and his sisters were Katharine Gray (wife of Hackley Bartholomew Bacon) and Frances Susan Gray. He was of French-Huguenot and Polish descent. In 1858, when he was twenty years old, he graduated from the
University of the City of New-York (which gave him an honorary D.D. in 1876), followed by preparations for the ministry at
Alexandria Theological Seminary in Virginia from 1859 until 1861, but due to the
U.S. Civil War, the school was relocated to Philadelphia, where he completed his degree. ==Career==