During the Middle Ages, fear of the Saracens marked the coastal landscape of Lazio as so-called "Saracen towers" and small fortress were constructed along the coastline. Among those, Torre Astura ranks among the most beautifully preserved.

The fortress's history begins in the 12th century, when it was owned by the powerful Counts of Tusculum, until it was taken over by the equally powerful Frangipane family.

The last of Hohenstaufen dynasty, Conradin, following the disastrous defeat at Tagliacozzo in 1268, sought refuge here among the Frangipane. They were eventually betrayed and handed over to Charles of Anjou and later killed. 

In later centuries, the castle was owned by the Caetani, the Orsini, and the Colonna, who encircled the pentagonal tower with walls. It was then connected to the land via a small bridge that can still be walked today (although access to the interior is not allowed).

There is a deeper layer to be uncovered at the site. The more careful observer will note Roman ruins, partly underwater, slightly emerging from the sand, framing the coves and beaches near the castle. They are the ruins of a great late Republican/early Imperial Roman villa, with great fish ponds and massive structures in opus latericium. It's believed Cicero once owned a villa here, and was one of his stops while escaping Marco Antonio.