That is correct. You can only go to the Concierge Lounge. Amenity-wise it is the better of the two. The Concierge Lounge serves full breakfasts and has a lunch menu too. You can't get that in the Diamond Lounge. I think I ate breakfast there 5 out of 7 mornings on our Oasis cruise a few months ago.
The evening cocktail hours are the same in both lounges.
Here is my order one morning. This is for one person, the 'ultimate' breakfast or something like that. Lox, bagel, fruit, muesli, omelet, bacon, hashbrown potatoes.