yes, you can leave the ship at all times while it is in port.
However, once the ship has docked (or once it has arrived at the anchorage) it must be cleared by the local authorities first. This is usually a fairly quick affair, but very rarely it takes somewhat longer. Nobody will be allowed to leave the ship prior to this.
Also, on most ships you must be back on board at least 30 minutes prior to the scheduled departure (few ships require you to be back 60 minutes prior to departure).
Tenders run throughout the day if the ship is at anchor (usually every 30 minutes or so during the midday "off times"). The first couple of tenders are usually reserved for shore excursion groups. Also, it might be necessary to get (free) tender tickets on board. The daily newspaper will include all this information.