John Blair Smith was born in Pequea, Pennsylvania Colony, the son of the Rev. Robert Smith, who ran a celebrated academy there. Like his older brother, John Blair Smith was valedictorian of the Class of 1773 at the College of New Jersey, now Princeton University. He was in the graduating class of 1771, with William Graham, the rector of Liberty Hall Academy and Henry "Light Horse" Lee, governor of Virginia and father of Robert E. Lee. He was recruited at the age of 19 to come to Virginia as a tutor at the new Hampden–Sydney College, then being founded by his elder brother, Rev. Samuel Stanhope Smith.