People do not affect your personality. It’s how you handle those people in general which is a reflection of who you are.
I say - better push people away than let others push you.
When I started developing a larger ego, some friends have started to diminish contact with me. I lost it all, but with that, I have noticed how much they don’t care about me