I think you would be best getting a taxi to it!
We went to naples last year but not to Capri, all I would say is be on your guard. A lot of taxis are waiting at the port, and they are very aggressive with each other for the business. We agreed a price before we got in the taxi to take us to the train station, to go to Pompeii, and he tried to change the price half way through the journey. Keep your valuables close to yo, several people were mugged in Naples, like watches cut off their wrists, and money taken from cash points. I must admit next time I think I would do a orgainsed tour for this port, it is the only place I have ever been to in Italy that scared me. Don't want to put you off, just wanted to give you a word or warning
Best Regards Kerry