The priory church of St Nicholas and St John stands on the hill at Monkton, across the river from the castle. Arnulf de Montgomery founded the priory in 1098 and gave it to the Benedictines. Much of the Norman nave survives; the choir was restored in the nineteenth century.
Next to it, Monkton Hall is thought to be the oldest house in Pembrokeshire, possibly the priory's guesthouse. The walk over from the castle gives a fine view back at the castle walls.