There were several versions of the Carlton route starting from August 1886 when the
Toronto Street Railway created a
horsecar line of that name. All versions traveled along Carlton Street but had a variety of terminals, and none of the earlier versions of the route had the east–west breadth of today's route. The route number 506 began being displayed on the streetcars'
destination signs on February 4, 1980, when the Carlton route was still operated entirely by
PCC streetcars. From February 18 to September 2, 2018, streetcars along the 506 Carlton route were replaced by buses due to a streetcar shortage From late June 2020 until the end of 2020, regular streetcar service along the entire 506 Carlton line was temporarily replaced by buses to accommodate several construction projects, including the modification of the streetcar overhead between Bay Street and High Park for pantograph use. Prior to July 27, 2024, the 306 Carlton night route terminated at
Dundas West station, while 506 Carlton terminated at High Park. On that date, the 306 route was modified to terminate at High Park; thus, both 506 and 306 covered the same route. This change coincided with the introduction of the 305 Dundas night route that, like
505 Dundas, terminates at Dundas West station. ==Sites along the line (from east to west)==