The settlement was originally named "Cold Water" and began as a log house that was used for religious services. An early settler, Elisha Tunnell, erected a church at that location known as "Tunnell's Chapel". The settlement was renamed "Colfax" in honor of United States Vice President
Schuyler Colfax. The Colfax post office was located at the settlement in 1870 and remained in operation until 1905, after which mail was delivered from Canton. The donated land had also been the location of two local churches that hosted Methodist and
Cumberland Presbyterian congregations in 1885. The location was also the site of revivals and summer camp meetings. Colfax had a population of 30 between 1885 and 1895 and grew to around 100 in the 1930s. Residents grew
fruits and
vegetables and the community had three churches, a cemetery, and several businesses and houses at its peak. It declined to 28 in 1968 and grew to 35 from 1974 through 2000. Another early settler named Kellam donated of land, and on part of it, a cemetery was established. The first burial, in 1884, was of Elisha Tunnell. Colfax had a
Masonic Lodge (number 904) in 1919. During the 1920s, Colfax had a boys basketball team which competed against local teams. ==Geography==