I have the exact same fear. But then, sometimes new friends appear where we don't expect them. And if everyone feels that way (or many people), isn't that proof that we have something in common? If noone wants to be left alone, isn't it okay to be afraid and comfort each other? Just some thoughts which help me when I get upset
I believe that we are part of something bigger somehow, and that the love we give comes back to us at some point. But then, I have my moments when I doubt that and lose hope.
Anyways, have a hug and a little comfort for your heart from me