During the Middle Ages, Rovigo was a small town in a territory often contested by the numerous local powers in Northeastern Italy, most notably Venice and Ferrara. A castle was built in Rovigo between the 12th and 14th-centuries by the Este family of Ferrara who controlled the town at the time.

The castle was built along the Adigetto river that passed through Rovigo but was diverted in 1937. Being positioned in the middle of the swampy territory of Polesine, the castle had a natural protection and became a place of strategic importance. The castle was fortified and expanded in the following centuries and was often under siege. Most of the fortifications were demolished during the 19th and 20th-centuries. Today, all that remains of the castle are these two towers and only small portions of the walls.

The tallest tower is known as Torre Donà and reaches a height of over 168 feet (51 meters), a remarkable scale for a medieval tower. The smaller tower is Torre Grimani and stands at only about 70 feet (21 meters) tall because its upper portions collapsed over time.

The terrain on which the castle was built was marshy and probably not well suited to support the structure. The subsidence of the ground under the weight of the two brick towers caused both towers lo lean and the Torre Grimani to collapse.