2020 census As of the
2020 census, Madison had a population of 2,129, with 595 families residing in the town. The median age was 46.2 years. 18.5% of residents were under the age of 18 and 25.0% were 65 years of age or older. For every 100 females, there were 89.9 males, and for every 100 females age 18 and over, there were 86.3 males age 18 and over.
2000 census At the
census of 2000, there were 2,262 people, 972 households, and 626 families in the town. The population density was . There were 1,056 housing units at an average density of . The
racial makeup of the town was 97.32% White, 1.54% African American, 0.31% Native American, 0.35% Asian, 0.13% Pacific Islander, 0.59% from other races, and 0.75% from two or more races. 1.41% of the population were Hispanic or Latino of any race. Of the 972 households 25.5% had children under the age of 18 living with them, 45.3% were married couples living together, 15.5% had a female householder with no husband present, and 35.5% were non-families. 31.5% of households were one person and 16.5% had someone living alone who was 65 or older. The average household size was 2.33 and the average family size was 2.92. The age distribution was 22.2% under the age of 18, 7.7% from 18 to 24, 27.0% from 25 to 44, 24.8% from 45 to 64, and 18.3% 65 or older. The median age was 40 years. For every 100 females, there were 88.2 males. For every 100 females age 18 and over, there were 83.7 males. The median household income was $38,955 and the median family income was $36,429. Males had a median income of $32,109 and females $21,379. The per capita income for the town was $19,494. 10.5% of the population and 8.2% of families were below the
poverty line. Out of the total population, 14.5% of those under the age of 18 and 2.7% of those 65 and older were living below the poverty line. ==Notable people==