Rothesay Castle stands in the middle of the town, a ring of curtain wall strengthened by round towers and surrounded by a broad water-filled moat. It was built by the Stewart family around the beginning of the 13th century, and its circular plan is unique in Scotland. In the 1230s a Norwegian-backed fleet took it after three days, the attackers hacking through the wall with axes; the saga account is the earliest recorded assault on a Scottish castle.
Robert III made it a royal residence and in 1398 gave his son the title Duke of Rothesay, still borne by the heir to the throne. The castle fell into ruin after the 17th century and was repaired by the Marquesses of Bute before passing into state care. It is managed by Historic Environment Scotland; access to the interior has been restricted in recent years, so check before you go — the exterior and moat are impressive on their own.