King Henry III ordered the construction of this castle near the southern coast in 1249 as added protection against a French invasion, a constant threat to the English during the 13th century. It served as a defensive outpost for many years and was greatly damaged in 1377 after a raid by French forces.

The castle was sold to Jean d’Ypres, a cloth merchant, in 1430. Despite selling the castle two decades later, d’Ypres’s namesake stuck with the structure. Several different owners would pass through the castle’s doors before it was purchased by the state. It was then utilized as a jail and courthouse. 

In 1743, the Deputy Mayor of Rye, Allen Grebell, was attacked and killed by a butcher named John Breads. Breads had been fined by the actual mayor for defrauding customers a few years earlier. In a fit of rage, Breads stabbed Grebell to death after mistaking him for the mayor.

Breads’s actions made national news, and he was subsequently imprisoned in Ypres Tower. After his execution, his corpse was placed in a metal frame called a gibbet and placed on public display. For more than 20 years, his body and skeleton rotted away in the gibbet. A similar gibbet is now on display inside the castle, complete with a replica skeleton. 

Additional structures were added to the tower in 1837, including a woman’s prison tower and courtyard. The new prison tower contained four cells in cramped, tight conditions. 

Today, the tower plays host to a small museum, complete with old weapons and exhibits covering various topics, such as smuggling and maritime affairs in Rye.