You are not insignificant. Since you moved 5 years ago, maybe you are very sensitive to people's reaction to you. I am that way (even without moving to a new area). I think I sometimes judge a potential friendship before it even begins and give up on it if I see any sign of disinterest. I have to learn to stick with it even when I don't feel I fit in.
I hope you find an answer.
Welcome to the forum!