Oh I agree, it's not just you. I've struggled with this for many years and have preferred to be a loner. In my mind it beats being around people who are ignorant, close minded, superficial, unoriginal, etc, etc. In general, I can say that I have an immense dislike for people. I've always been different in my thoughts, beliefs and in my interests...finding it difficult to find another who is on the same page or wavelength as I.
I have found one or two people who I truly connect with but they are across country. Hopefully you will find a friend or two one day soon that you will have a connection with. Until then, just know you are not alone
-Jess