I am introvert too and it is hard for me to make friends. I have very few of them, who I meet once every two months or something. I like spending time with myself. It is a recovery time for me. I have few of hobbies which I enjoy on my own. It is hard to enjoy them with other people anyway. You can not draw with others a painting for example.
Some of my friends call me antisocial, some of my relatives have told me that many times I keep distance fron people and they say it in a negative way, that I am not very approachable (even when we are out together). I don't mind at all for all of this, as I recognize that it is true haha and they still love me. I am not very much interested in "fashion" activities that they follow: clubing, dancing, partying, multiple people "sexing", so although sometimes I seek for company, I am not interested in following people to all of that stuff. So yes, mostly I am a loner, by choice but also in luck of people with common life style.
And I don't know what else other people might say behind my back... I haven't heard them