Harvey and Grant built and lived in
Glen Oaks, a 27–room mansion in the center of LaFollette designed by Knoxville architect
George Franklin Barber. The home on Indiana Avenue is listed in the
National Register of Historic Places. As his company grew, he traveled extensively and remained close to his extended family. During the glare of the
Wisconsin governor's race of 1900, his mother and older brother and sister returned to Primrose with cousin Robert and other family members to visit the old log cabin where Harvey was born and to relive memories of the pioneering days of a half century earlier. During the next decade, Harvey and his wife, Katherine Warner (LaFollette) Cooper were often in Washington, DC, visiting family members and meeting political allies. Vice President
Charles W. Fairbanks was a friend from his younger days in Indiana. His brother and cousin both moved to the nation's capitol as members of Congress, and he visited them often. Later in his life, when the mines no longer produced and his business failed, he returned to his academic roots, corresponding with
Abraham Lincoln scholar Louis A. Warren, who wrote
The Lincoln and LaFollette Families in Pioneer Drama, a history of the Lincoln and LaFollette families on the Kentucky frontier in the years before
Jesse LaFollette and
Thomas Lincoln took their families across the
Ohio River to settle in
Indiana Territory. ==Family==