"Come On Baby Let's Go Downtown" was included in the set list of
concerts that Young was performing with Crazy Horse at the
Fillmore East in March 1970 with Whitten taking the lead vocals. Young had recorded 9 songs for the ''Tonight's the Night'' album in 1973 but did not feel the album was finished, and so the album sat unreleased for two years. Young's manager
Elliot Mazer suggested adding three older songs to the album - "Come On Baby Let's Go Downtown" as well as "Borrowed Tune" and "Lookout Joe." "Come On Baby Let's Go Downtown" was released as a single from the album.
Allmusic critic Matthew Greenwald stated that this version is "a fabulous country barnburner, which encapsulates Crazy Horse's loud-and-proud appeal." He also noted that it was a sensible song to include on the album since it deals with drug culture but "with a lighter touch than some of Young's songs on the album." Although ''Tonight's the Night'' producer
David Briggs felt that the material added to the original album recordings detracted from the mood of the album, Young biographer Jimmy McDonough disagreed, saying that "when gone-dead Danny Whitten's voice jumps out of the speakers singing 'Come On Baby Let's Go Downtown,' it just hits you in the gut that much harder." Music journalist
David Downing called it "great rock 'n' roll," saying that Whitten singing it in this context gives the song added resonance. Young and Crazy Horse performed "Come On Baby Let's Go Downtown" live on their
Rust Never Sleeps tour, with Young taking the lead vocals. ==Cover versions==