Boarding time always depends on how quickly or slowly the previous cruise is processed by Customs and debarked. Also, it's possible that other things can happen that delay boarding, such as a surprise Coast Guard inspection. All things being equal, however, boarding CAN start as early as noon, or as late as the stated boarding time.
If you go to the pier from your hotel, you may be able to board early. If not, they'll have you seated until boarding time, assuming the seating areas are not filled by the time you get there.
You could also try asking your hotel for a late check-out on cruising day, so you can relax at the hotel before traveling to the pier.