It sounds like the T/A you spoke with either had several passengers on the maiden voyage of Quest or has a completely different demographic than many of us. There were major problem on the first couple of sailings for these ships when they were in the transition period between Celebrity and the new branding and refurbish that became Azamara. I had some folks with issues too, there were mechanical and staffing problems just to name a couple. These issues were quickly addressed and compensated for with those who were unfortunate enough to experience them. Since that time not only have there been no further problems that I'm aware of, other than the usual handful of people that cannot be pleased and they're usually pretty easy to identify. Both ships have had among the best reports back of any lines out there including some of the 6 star offerings. I have overcome my fear of recommending Azamara after their rocky start and am happy I did. I understand the T/A's reluctance at referring you if it was duing that time frame but am willing to bet if you still have contact with her and were ask that same question today you'd get an entirely different answer.
Either choice would be a memorable cruise with among the best in the industry. If I were choosing between the two for myself, I'd probably book Azamara if the itins were similar or just as desirable on each.