The town was first settled around 1720 in
The Oblong, which was a disputed area in southeastern
Province of New York also claimed by the
Connecticut Colony. The Oblong was a strip of land approximately 1.81 miles wide (2.91 km) between Dutchess County, New York, and Connecticut, ceded to New York in the 1731 Treaty of Dover. Between 1720 and 1776, a large number of mostly Connecticut families settled in the southern Oblong. They could not settle west of it because that land was privately owned by the
Philipse family. It had been granted a patent for virtually all of the remainder of the area of the future Putnam County. The first such settlers in the Oblong were the Hayt family, who built a house at The Elm in 1720. Another early settler was Jacob Haviland, who settled Haviland Hollow in 1731. The first village in Putnam County, the hamlet of Patterson, was originally called Frederickstown. The eastern part of the future Putnam County was called Southeast Precinct (not the same as the current town of Southeast). The Philipses were
Loyalists during the Revolution and left the area. The state confiscated their land, selling it off. In 1788, the former Philipse portion of the Oblong was chartered as the Town of
Southeast; the remainder of the region was chartered as the Town of Fredericktown in
Dutchess County, New York. In 1795, Fredericktown township was split into four parts: 1. the Town of
Carmel, 2. a part which was combined with the northern half of Southeast and became the Town of Franklin, which was renamed the Town of Patterson in 1808, 3. a part which was combined with the southern half of Southeast and became the new, much larger town of Southeast, and 4. the remnant of the town, which was the Town of
Frederick for a while. It was renamed as the Town of Kent in 1817. ==Notable people==