Handling the public is a skill - you are learning and will get better and better at dealing with disagreeable people - courses are often offered on 'dealing with difficult people' - perhaps your employer would send you on one if you say you want to improve your customer service skills.

Of course no sensible person gives a toss about improving their customer service skills but it will sell the idea to your manager better than the idea that the customers hack you off so much you need help dealing with it!