The De Wauncys were amongst the earliest possessors of this manor since the
Conquest. From them, it descended to the Brookes. On 24 April 1471, Thomas Lovett II acquired Astwell in exchange for his hereditary estates by a family arrangement with his mother's cousin, Dowse Billing and her husband William Brooke. The house was begun by Lovett. George Shirley, Esq. (created a baronet, on the first institution of that order, by James I. in 1611), Lovett's grandson, was the next possessor. He partially or totally rebuilt the manor house. From him this manor descended to
Washington Shirley, 5th Earl Ferrers by whom it was sold in 1763 to
Richard Grenville-Temple, 2nd Earl Temple. He was succeeded by his nephew,
George Nugent-Temple-Grenville, 1st Marquess of Buckingham, and whose son,
Richard Temple-Nugent-Brydges-Chandos-Grenville, 1st Duke of Buckingham and Chandos was also associated with the manor house. Biddlesden Abbey had possessions in the manor. By 1874, the manor house had been largely dismantled, some of the "inferior offices" becoming a farmhouse, and a "broad embattled tower" remaining. The castle has been Grade II Listed since 1953 (List Entry Number: 1041132) but the report states that "in 1958 the building was restored for Mrs. Joan Wake". News items in 2019 reported that the mansion "retains fine features with mullions, window seats and leaded lights, fireplaces and an oak staircase." The building, with gatehouse tower that includes a spiral staircase leading to rooms and to the castellated roof, offered four bedrooms and three reception rooms; it was said to be habitable but additional restoration was desirable. ==Architecture and fittings==