We walked to the Little White House from the ship. It's a couple of blocks from Mallory Square. There is an admission fee, but then tours that take about an hour are included. I suggest a tour, as you get a lot more information than walking around the house on your own. You can walk into the Museum Shop and buy tickets, but there may be discounts if you buy online, if you're sure you want to go.
See more information here.