I think we're on the same page. The operative word is HAVE.. you don't have to let them matter. Caring for people should be a choice, don't you think.
But, by living by this golden rule, if I am doing my best, and I know this, and with God as my guide, then how others perceive me isn't about me anyway, right? People's opinions of me, while appreciated if positive, aren't really my business, it's theirs.