You don't say which port and that is important.
Miami is the strictest I have seen. (but that is because the complex is a good size and the non VIP cards are on the other side of the complex.
Smaller terminals like New Orleans...the non VIP check in desk is 3 steps from the VIP office.
However, the official rule is that the only folks who get to go through VIP are:
people actually in YOUR cabin,
minor children in another cabin,
or any individual in another cabin that has their Sail and Sign card covered by YOUR credit card.
Make sure your TA or PVP has linked your cabins so that you get the same dining time and table.