In 1513,
James IV of Scotland invaded England again with a powerful army that included artillery. He crossed the border and moved on Norham. His guns pounded the outer defences for several days until the outer ward was taken by the Scots. Soon after, the castle surrendered. By then, most of the outer walls had been destroyed. Weeks later, James was defeated and slain at the
Battle of Flodden, near
Branxton in Northumberland, and Norham fell back into English hands. The castle was inspected by
Cardinal Wolsey's chaplain William Frankelayn on 29 August 1515. He found it "well fortitified with
contremurs and
murderers." The wall from the "dungeoun" (the Keep) running south west towards the kitchen was 28 feet thick. At this time, Scotland's
Regent Albany was planning to bring an army against the
Hume family on the Scottish borders. Bishop
Thomas Ruthall of Durham inspected the castle and began restoration work, which continued until 1521.
William Dacre, 3rd Baron Dacre (c. 1493–1563) was Captain of Norham Castle in 1522–23. During another invasion scare from Albany, in September 1523 the
Earl of Surrey, Frankelyn and Sir
William Bulmer,
Sheriff of Durham, viewed the defences. Surrey gave orders for new earthwork defences of platforms and rampires and the "mending of broken places with turvis and yerthe." He estimated the work would take six days and made similar orders for the repair of
Wark Castle. The steward of the
Earl of Northumberland, Roger Lascelles,
parleyed with the
Earl of Angus and William Douglas,
Abbot of Holyrood, across the Tweed on 5 September 1528. Angus was threatened by his step-son
James V of Scotland and he asked Lascelles for chambers in the castle to be provided for his daughter
Margaret Douglas, her governess
Isobel Hoppar and the young
Earl of Huntly. If necessary Angus himself would seek safety in Norham. Margaret Douglas, the grandmother of
James I of England, was received at Norham in October. The Earl of Northumberland assured
Cardinal Wolsey that members of the refugee Douglas family would not be allowed in the inner wards of the castle. The castle was maintained in a state of good repair, with a strong garrison during the remaining conflicts with Scotland in that century.
Brian Layton, Captain of Norham, reviewed the security of the castle in October 1542 after hearing rumours that insiders would hand the castle to the Scots. Layton invaded Scotland during the war of the
Rough Wooing and was killed in February 1545 at the
battle of Ancrum Moor.
Sir Richard Lee with
Sir Thomas Palmer and Sir
Robert Bowes surveyed the castle in 1550 and in 1554
Bishop Tunstal carried out repairs. However, when an extended state of peace existed between the two countries, the garrison was reduced and the defences were allowed to deteriorate.
Lord Hunsdon had the floors of the hall, parlour and kitchen propped up with ship's masts in 1574. By the end of the century the castle had already fallen into a state of disrepair. In 1596
Queen Elizabeth gave the Captain,
Sir Robert Carey, her “resolute answer” that she would spend nothing on Norham. Norham was the site of regular meetings of border wardens. A meeting at the west ford of Norham in October 1597 ended in a gun fight over the River Tweed at dusk. The castle was destined not to see any further fighting, but the castle and manor were given to
George Home, 1st Earl of Dunbar, by
James VI of Scotland on his accession to the throne of England. The castle was left to fall into ruin. ==Nineteenth century==