You describe these people as "significant," but they are not that significant to you now, are they? I mean, you describe them as "fighting" you, but the fact is you have moved on with your life and moved to a different place. People change, friendships change or evolve or lessen over time.

These people may feel threatened by your lack of need for them, or rejected, or hurt, or whatever. But the important thing is that you take care of you. You say you don't know how to deal with this, but I think you do know. You do as you have done, and avoid being around these people as much as possible.

I wonder about your one friend and whether this person is as different as you think. Why is he telling you this stuff that people are saying? It doesn't sound very supportive to me. If you want to hang out with him just say, "I'm not so interested in what they think, thanks," and move on to a different subject.

Therapy seems like a good place to explore why the opinions of these people seem to matter so much to you, even though you don't feel your friendship with them was authentic.