We took a bus to Honfleur and it was very reasonable and very convenient. The tourist info office right at the port gave us the information we needed, complete with a bus schedule. We took a taxi to the bus station, rode the bus for 30 minutes, spent a great day walking around and took another bus back. There were about half a dozen other people from our ship on the same bus.
We arrived in LeHavre on a Sunday so there was no staff at the bus station but we bought our tickets from the driver.