Far from becoming a recluse, immediately after the duel, Byron planned a holiday to the
Austrian Netherlands town of Spa with his wife and sister, Isabella, Lady Carlisle. The latter noted that the couple were unable to control their spending, calling them "ye worst managers I ever saw". Sometime in the late 1760s, Byron schemed to resolve his serious financial difficulties by marrying his son and heir
William into a wealthy family. But just before the marriage William eloped with his cousin Juliana Byron, the daughter of Byron's younger brother, the
naval captain and later
Vice-Admiral John Byron. Despite the myth that Lord Byron became enraged by his son's elopement and subsequently tried to wreak revenge by ruining his inheritance – tearing down trees, selling off artworks and killing over 2000 deer – the fall of the estate in fact came about because Lord Byron simply could not pay his debts without the cash injection through an affluent daughter-in-law. The real neglect of Newstead did not occur until after the death of Byron's son in 1776 – this left Byron legally unable to sell off parts of his lands and estate without the permission of an adult heir (his grandson was not due to turn 21 until 1793). In around 1778, he was forced to sell off the majority of his effects at Newstead Abbey, in what became known as the "Great Sale" – this included artistic masterpieces, hunting gear, furniture and even toothpicks. His wife Elizabeth left him shortly afterwards, taking their only surviving child Caroline with her. Caroline died in 1784, leaving the couple childless. Later accounts attest that he subsequently took one of the servants, Elizabeth Hardstaff, as his mistress and that she became known as "Lady Betty". Though there is no solid evidence of a relationship, she was one of the few people mentioned in his will. ==Death and legacy==