In Cozumel, you will probably have to take a cab to one of the beach clubs. They are right at the pier when you get off the ship, and there will be cabs waiting at the beach clubs to take you back to the ship. The one beach club I went to that I liked was Paradise Beach. I think it was about a $15-20 cab ride from the ship. They take US dollars in Cozumel, so don't bother exchanging your money.
There is no entry fee for Paradise Beach, as the impression is you'll spend money while you're there. Lounge chair use is $2 US, and use of their aqua park (large inflatables that they put in the water) is $12 US. There is a restaurant on the premises as well for lunch, and for drinks.
I can't say anything for Belize, as I haven't been there.
If you don't want to venture out on your own, then maybe check
www.shoretrips.com, which offers shore excursions for maybe cheaper than the ship.
__________________
2007 - Majesty of the Seas
2009 - Enchantment of the Seas
2010 - Freedom of the Seas