The building is set on the main beach of the town, which lies on the rugged coast of the
Bay of Biscay at the foot of the
Pyrenees. In 1854, the emperor
Napoleon III and his wife Eugénie bought several acres of dunes in Biarritz and gave the engineer Dagueret the task of establishing a summer home surrounded by gardens, woods, meadows, a pond and outbuildings. Napoleon III chose the location near Spain so his wife would not get homesick for her native country.
Hippolyte Durand was given the job of designing the Villa Eugénie and outbuildings, and overseeing construction, which began in 1854. Durand was abruptly dismissed in June 1855 and replaced by the twenty-seven-year-old architect
Louis-Auguste Couvrechef, who adopted a less austere style. Couvrechef died in 1858 and was succeeded by the architect
Gabriel-Auguste Ancelet. Ancelet built a new wing facing the lighthouse before he was replaced by
Auguste Lafollye, who added an attic with rooms for the staff. The villa, in néo-Louis XIII style, consisted of a long main building with three perpendicular wings. The first two wings formed a court, while the third was shorter. A description in 1856 said that the two-story villa left much to be desired for such an important residence. It was excessively simple, with so little in the way of ornamentation that the facade looked bare. Later the facade was decorated with busts of members of the imperial family and regional personages.
Victor Huguenin undertook the decoration with Corinthian columns and reliefs of arms and crowned eagles. In 1830,
Victor Hugo had expressed the hope that Biarritz would remain unspoiled. However, when the imperial family established its residence, it quickly became popular as a resort with the wealthy and aristocratic. The imperial family spent most of August and September at the villa each year, with a small entourage. Early visitors were not always impressed. An English magazine described the village in 1858 as: ==Development of Biarritz==