The castle was originally built in the 12th century as a defense against attacks from the sea and coastal road and went through several constructions and ownership changes throughout the 13th century. William de Moin built the castle in 1066.