There has been a castle of some sort on this site since the early 13th century.