Lee was born on April 9, 1804, at
Tremont Place in
Boston, Massachusetts, and named after the Rev. John Clarke, D.D. He was the son of Nathaniel Cabot Lee (1772–1806) and Mary Ann (
Cabot) Lee (1784–1809), who were first cousins. After his father's death in
Barbados, his mother married Francis Blanchard. His mother died at 25, shortly after giving birth to his only sibling, half-sister Elizabeth Cabot Blanchard, who also died young at age 33 in 1842 after becoming the first wife of
U.S. Senator Robert Charles Winthrop and having three children. His maternal grandparents were Anna ( Clarke) Cabot and Francis Cabot (brother of
U.S. Senator George Cabot), who lived in
Natchez, Mississippi. His paternal grandparents were Capt. Joseph Lee and Elizabeth ( Cabot) Lee (daughter of Joseph Cabot and Elizabeth Higginson Cabot). After losing his father and mother by the age of five, he went back and forth between
Wenham with the family of the Rev.
Rufus Anderson and the Pickering family, and
Duxbury after the death of his step-father in 1813. He also spent time with his great-grandmother, Sarah Pickering Clarke (widow of Capt. John Clarke and sister of Col.
Timothy Pickering). He later went to Salem where he entered the private school of
Abiel Chandler and John Brazer Davis, where he prepared for college. Lee entered
Harvard College in 1819. He graduated with the class of 1823 but did not collect his diploma until 1842. ==Career==