Heya, CherryGlazer, welcome to PsychCentral.
Since most of the time I feel like nobody notices me, comments as you describe seem like a long shot and well out of my experience. I can understand how it would degrade one's sense of trust in honest human relations.
Without diminishing your experience, I want to say that it's an action and attitude coming from them and not really reflecting you. Be true to yourself and don't let it bother you.