Olsson started playing handball at the age of 12 in
Lugi HF in Lund. It was also here he started his career in 1964 when he was only 16 years old. Lugi was at that time the Swedish 2nd division, the second highest level in Sweden. The same year Lugi was promoted to the highest tier, Allsvenskan, and thus Olle Olsson made his top flight debut the next year. In 1970 he was in military service, and therefore he played a single year at
IF Saab, only to return to Lugi HF. He played a total of 136 matches, scoring 523 goals. In 1975 he stopped playing at Lugi to study at the
Swedish School of Sport and Health Sciences in Stockholm. Therefore, he started playing for the Stockholm club
AIK, where he was a playing manager. He played for AIK for 5 years, while he was studying. In 1980 he retired as player to coach full time. == National team ==