The castle was built either by
Alan fitz Walter, Steward of Scotland (died 1204), or by his succeeding son
Walter (died 1246). During Alan's tenure, the family added the
Isle of Bute to their lordship. A wooden castle was constructed first, but the stone circular curtain wall was in place by the 1230s, when the castle was attacked and taken by the Norwegian-backed fleet of
Óspakr-Hákon, King of the Isles (died 1230). According to
Hákonar saga Hákonarsonar, Óspakr's forces fought for three days to take the castle, breaking down part of the eastern wall by hewing the stone with their axes. This saga is the earliest recorded account of an assault on a Scottish castle. In 1263, Rothesay was taken again by the Norwegian forces under
Hákon Hákonarson, King of Norway (died 1263) before the
Battle of Largs. Although the Battle of Largs is called indecisive, Hákon's campaign was a failure, the Norse retreated, and it effectively ended Norwegian authority in western Scotland. The early castle comprised only the roughly circular curtain wall, thick, high and around in diameter, built on a low mound, with a battlement on top accessed by open stairs. The moat was connected to the sea, the shoreline then being closer to the north-east of the Castle than it is today. The broad
crenellations can be made out within the walls, which were later raised. Holes in the upper wall would have supported a timber bretasche, a projecting structure serving as an extended battlement. This curtain wall was built of coursed ashlar, and had only two openings in its length. The main gate was an arched opening with a simple timber door. The second opening was a small postern gate in the west wall, later blocked. In the later part of the 13th century, the castle was strengthened by the addition of four round towers, of which only the north-west survives intact. These three-storey towers had strong splayed bases, with arrow slits below the crenellated parapet. A
portcullis was added to the main gate. ==Wars of independence and the rise of the Stewarts==