My personal opinion (for what it's worth) is... no you're not wrong. Are you missing out on something? Yes you perhaps are. But, from my perspective at least, there's nothing to be gained by trying to force yourself to do things you're not comfortable doing. At least not all at once. If there are opportunities, on occasion, to try being just a bit more independently social that might be worthwhile. (After all, you do have a whole lifetime of potential social interactions to get through.) But I at least see no value in trying to make yourself do things you're not comfortable doing.

Perhaps it comes down to a matter of degrees. If you're just a bit uncomfortable perhaps it makes sense to put yourself out there. But if it really hurts, so to speak, why punish yourself? I personally don't think that's healthy. In years to come perhaps you may look back & wish you could have been more social. I don't know. But right now I think you have to do what you feel you can do within the limitations you have. My best wishes to you...