The club was founded in December 2001 as a merger between
Sør-Aurdal FK and the football branch of
Fagernes IL. It was known as
SAFK Fagernes until 2006, when the current name was implemented. The men's team had a run as a mid-table
Second Division team between
2008 and
2013. In the
2014 Second Division, the team ended last and faced relegation. In 2019, the first-team squad was almost completely disintegrated. The remaining team plummeted out of the Third Division, following lacklustre results such as a 0–19 loss. The team started with an outfield player as goalkeeper and often lacked substitutes. However, they were praised for sportsmanship in actually finishing the season instead of pulling out. The 2020 season was cancelled due to
COVID, but as the
2021 Norwegian Fourth Division progressed, Valdres ended last and was relegated to the sixth tier. Still, the team secured a berth in the
2022–23 Norwegian Football Cup first qualifying round. Valdres lost 0–3 to
Gran IL. The women's team plays in the Third Division. ==Season to season==