Castleberry showed extraordinary ability as a Georgia Tech "pony" back, standing at only and weighing only . At
Boys High School in
Atlanta, he averaged 171 rushing yards per game and scored 102 points. In
1942, freshmen were ruled eligible to play varsity ball as college rosters were drained by
World War II, which allowed Castleberry to display his spectacular prowess early. On October 3,
1942, Georgia Tech beat
Notre Dame for the first time since
1928, especially impressive as the game was played in
South Bend and Notre Dame had not lost a game in two years. Castleberry led Georgia Tech in both passing and rushing yards, even after Notre Dame head coach
Frank Leahy had been warned by a scout that Castleberry was "the most dangerous runner in America." Against national power
Navy, Castleberry led Tech to an impressive
shutout win on October 24 in Annapolis, broadcast worldwide via the
Armed Forces Radio Network. The momentum changer of the game occurred when Navy was driving for a score near the Tech goal line. Castleberry was put in to play defense on a passing situation on fourth down. He anticipated the Navy pass and stepped in front of the intended receiver for an interception. He scampered away with a dazzling run of 95 yards for a touchdown. Tech coach Bobby Dodd described it as “a magnificent run that I can still follow step by step in my mind. It shook Navy to its keel, and Tech won 21-0.” Dodd called it Castleberry's greatest play. The national press began to take notice of Castleberry. Georgia Tech won its first nine games and climbed to a
top five national ranking. During that ninth win, Castleberry injured his knee in a hard-fought 20–7 win over
Florida in Atlanta. Despite the injury, Castleberry played in the final two games of the season, but not quite at full speed, and Georgia Tech dropped both. On
New Year's Day, Tech traveled to the
Cotton Bowl in
Dallas, but lost a tight game to
Texas, 14–7. Castleberry's injury improved for the bowl game, yet he was still not quite a hundred percent and the team finished at 9–2. Castleberry was third in the
Heisman Trophy voting, behind winner
Frank Sinkwich of
Georgia and runner-up
Paul Governali of
Columbia, both seniors. This was the highest a freshman had ever placed in the Heisman voting. Castleberry's number 19 is the only football jersey Georgia Tech has ever retired. ==Army Air Forces==