Hi!
I just wanted to say I have a similar experience to what you describe when I'm talking to people or trying to make friends. It feels as though I am not as into what the other person or people are into, and therefor, I am not part of that social group. Like I am not enough of a fan to really relate to them well.
For example, I enjoy anime and manga, but when I go out to join up with my local anime and manga meetup group, it feels like they are all die hard fans and have seen many different shows, and I'm just a casual viewer and only have a few shows that I've watched. I therefore don't feel like I'm enough of a fan to really belong in their group, and I begin to grow more and more quiet and just go off in my own world. That's why I rarely go to meetups anymore.
I know it's probably just my own skewed perceptions, but it feels nice knowing I'm not alone in feeling things like this. I don't really know how to overcome this way of perceiving social situations, so I can't offer any advice, but I can offer my moral support and empathy.