Does anyone else feel like they never belong?

For as long as I can remember I've felt a lot like I don't really belong too much. In social situations I tend to feel as if I simply don't fit in with any of the people around me. This could be because I'm a bit shy or have a bit of a social phobia? But I don't know, I've always felt like even if I do gather up enough courage to talk to someone about my interests and what not, even if we do have a lot in common I simply don't really 'click' with anyone. I haven't had too many friends in my lifetime and the people I have befriended have eventually lost interest in being around me. So what's with me? Am I boring, stupid, annoying? I can never figure it out. I mean... there's obviously something up.
Has anyone else ever felt this way before?