Originally, a rectory was located on the site, which provided a home for the rector who officiated in the private chapel of St. Nicholas in the Beauchamp manor house nearby. When the rector, Henry de Wyk, resigned in 1304, the lord of the manor,
John de Beauchamp, 1st Baron Beauchamp, established a
chantry and founded a college for five chaplains. It is suggested in the
English Heritage listing that the rectory was probably already owned by the Beauchamps, but if not they purchased it upon de Wyk's departure. The college was composed of four priests governed by a
provost, and was attached to the manor house chapel. A
National Trust report referenced in the Historic Environment Record could not determine if the pre-1304 rectory was retained for the college's use, or if a new building was erected instead. In 1444, a report by the
Bishop of Bath and Wells,
Thomas Beckington, recorded that, due to poor management by the provost, the college had fallen into disrepair: the buildings were in a bad state, and only two priests remained alongside the provost, rather than the intended four. Beckington gave the governance of the college to the
precentor of
Wells, and a few years later the current building was erected. In his article, ''Chantry Priests' Houses and other Medieval Lodgings'',
W. A. Pantin suggests that this building is probably smaller than the original college, due to the reduced number of clergy, and that it was most likely erected around 1450–60. From 1473 to 1508 Provost Coorte was in charge; however for 36 years during that time he was non-resident. The college was converted into a private house, and underwent significant renovation. Pantin suggests that the building work can be approximately dated to the period in which the Strode family inhabited the house, based upon an inscription in some panelling, "1585/TS", in which the TS stands for Thomas Strode. three barns (one of which is ruined), a ruined outbuilding and a dovecote. ==Architecture==