We always return to the ship early because this is when it isn't crowded. It is quite relaxing to have the pool almost to ourselves. For tender ports we usually find no line and walk right onto the tender.
This is when I take photos of the ship too. The lack of passengers makes it easier.