I looked at the various ports from satellite view and they don't look to be hubs of activity. I reckon people don't want to live around ports so the richer areas will not be there. The Baltimore port isn't around anything either so I guess that's true for most. Although, the St. Thomas port was right by a shopping center. For the most part though, it seems like you have to travel to get to stuff.
I looked at the La Paz Waterfalls Garden site and that looked nice. But I've been deceived by Web sites before so I don't know.