Well, if you want a beach day and visit to Margaritaville, then I would take a taxi to Doctor's Beach (within walking distance of Margaritaville) and then visit Margaritaville. Shopping downtown is not as bad as you have heard. It is only bad if you respond with an arrogant/aggressive attitude. Jamaicans are wonderful people, but different than you. If you give them a little respect they will give you a lot.
If you want to tour Mo Bay for the day, I would suggest contacting Trevor Hudlin (firstname.lastname@example.org) and have him put a tour together for you. He is great!
I hope that helps