Annapolis is a small port and home of the US Naval Academy. If the quiz is going to include Olympia, Annapolis is far easier to navigate to by boat from the closest ocean.
Olympia and Annapolis are pretty big stretches. I doubt anyone that just took a dip in the Chesapeake or the Puget Sound says they swam in the ocean that day.
Sounds and bays are considered to be extensions of oceans by definition, no matter how far they stretch into land. Thus, if any state capital were on the Gulf of Mexico, it would be on an ocean (i.e., the Atlantic).
I certainly would and I've been all over the ocean. The Chesapeake's at sea level full of ocean creatures. It's not a river, you can't see across it in places, it has dangerous storms. It's no different than Long Island Sound.
I think it needs to be a body of water a little more prominent than a river; otherwise, there are a host of other capitals that should count if the criterion is just being on a river than leads into the ocean - Augusta, Hartford, Albany, Richmond, Baton Rouge, St Paul (as someone above pointed out), maybe more. I thin it's Narragansett Bay that counts in the case of Providence.
The Providence River is not a river in the usual sense, as it is a tidal river formed by the confluence of two rivers. This In that sense, the Providence River is more an extension of Narragansett Bay (and thus of the Atlantic Ocean) than it is a conventional river (cf. Albany on the Hudson River). Admittedly, in the end, Providence's oceanic bordering is the most questionable on the list, not Annapolis' or Olympia's.
Providence is in very plain sight in Narragansett Bay. The rivers to Providence and Fall River aren't great fresh water rivers, they're more broad inlets from the ocean at the end of very short, minor rivers. Augusta and Hartford are way up proper, heavy flowing rivers that have almost zero brackishness. Augusta gets a minor tide as the lower reaches of the Kennebec River get backed up by the Gulf of Maine's immense tidal surge.
Alrighty, I'm not from the USA, however, based on this argument for Providence, can someone explain why New York City is not on here, just out of curiousity?
Here's a fun fact: Washington's capitol building is the only one that is on a waterfront. There's a great picture you can take of the dome reflecting in Capitol Lake, which used to be an inlet of the Sound until it was diked.