Richardson returned to federal politics in 2004, winning the election in the electoral district of
Calgary Centre which had previously been held by PC Leader
Joe Clark. He was re-elected in 2006, capturing 55.4% of the vote, in 2008 with 55.6% of the vote, and in 2011 with 57.7% of the vote. Among his parliamentary duties, Richardson first served on the Environment Committee where he became the only Conservative to win the
Sierra Club's "Green Parliamentarian" award. Richardson was appointed Chair of the House of Commons Standing Committee on Natural Resources in 2006. From 2007 until 2011, Richardson served as the Chair of the International Trade Committee. According to the
Montreal Gazette, "the International Trade Committee, under the genial chair of Conservative Lee Richardson, is a collegial exception to the toxic tone of most House committees." The International Trade Committee under Richardson chairmanship successfully vetted three free trade agreements. The committee went clause by clause through and referred back to the House the following trade agreements; Bill C-2: An Act to implement the Free Trade Agreement between Canada and the Republic of
Colombia, the Agreement on the Environment between Canada and the Republic of Colombia and the Agreement on Labour Cooperation between Canada and the Republic of Colombia (Canada-Colombia Free Trade Agreement Implementation Act), Bill C-8: An Act to implement the Free Trade Agreement between Canada and the Hashemite Kingdom of
Jordan, the Agreement on the Environment between Canada and the Hashemite Kingdom of Jordan and the Agreement on Labour Cooperation between Canada and the Hashemite Kingdom of Jordan (Canada-Jordan Free Trade Act) and Bill C-46: An Act to implement the Free Trade Agreement between Canada and the Republic of
Panama, the Agreement on the Environment between Canada and the Republic of Panama and the Agreement on Labour Cooperation between Canada and the Republic of Panama (Canada-Panama Free Trade Act). After the 2011 federal election, Richardson was one of eight members of Parliament who put their name forward to be the
Speaker of the House of Commons. Richardson lasted to the fifth round and placed third. ==Resignation and post-parliamentary career==