Name a law, and at some point over the last quarter-century, it’s probably been broken within the walls of this living Manhattan relic. What’s now among the oldest remaining drinking establishments in New York City has been throughout the course of its sordid history a distillery, a brothel, a smuggler’s den, and a speakeasy. There’s allegedly a ghost named “Mickey,” and it was once home to James Brown—not that one.

The bar known today the Ear Inn was built as a home in 1817 for James Brown, an African-American who fought in the Revolutionary War and served as an aide to George Washington. He’s rumored to be the man rowing at Washington’s knee in Emanuel Leutze’s famous painting, Washington Crossing the Delaware. He commissioned the building with money made from a lucrative career in the tobacco industry. At the time, the shores of the Hudson nearly reached the walls of the building. A wooden panel on the bars facade indicates the late 18th-century’s high tide. Extensive landfilling would expand Manhattan westward into the river to accommodate a booming shipping trade.

As with most longstanding drinking establishments, the ensuing chain of events is far from definitive and hardly bears detailing. It should suffice to say that the building changed roles and hands, in no particular order, from a distillery serving passing sailors; to a bar whose upstairs was a functioning brothel; to an eatery that served construction workers building the Holland Tunnel; to a bar-restaurant whose upstairs was a doctor’s office; to a restaurant-speakeasy during Prohibition; and finally settling into the modern day as another (albeit, far, far older) Lower Manhattan dive bar. Over the years, the building earned a reputation for being a one-stop shop for every sailor’s unsavory needs. Today, resident ghost “Mickey the Sailor” is rumored to have drank himself to death here in the late 1800s, and makes appearances from time to time waiting for his next shipping job.

While the neighborhood around it becomes unrecognizable from the bastion of vice and recklessness it once was, the Ear Inn is one of the last gasps of edge in Soho. “It’s all dog walkers and joggers here now,” co-owner Richard “Rip” Hayman told The New York Times. “In the evening, you can smell the Botox.” Inside, however, the Ear retains the air of an 18th-century longshoremen’s den of vice, its walls crusted with gritty nautical curios and oddball, ear-centric artwork from decades of collection.

For much of its existence, the bar was informally known as “the Green Door” in honor of its entryway. It earned its new name in the 1970s. In order to give the spot a more official title without wading through a sea of paperwork to satisfy the Landmarks Preservation Commission, the owners simply spray-painted the right-half of the ‘B’ on the neon “BAR” sign out front, giving the Ear Inn its name.