You can get quick transportation to any part of the beach you chose to use; simply ask your driver to pick a spot with a restaurant/bar on the beach that rents lounges and you can hang for a while and veg out.
The bus station is only a couple of minutes from the port entrance
buses running along seven mile beach seem to be quite frequent
just make sure you leave enough time to get back to the port
before you arrive at Grand Cayman do as much research as you can.