I've had social anxiety since I was a toddler. Almost all my life, I never had panic attacks. My social anxiety was sufficient to cause me extreme sensitivity and isolation, as a child. I think that right there means it is a disorder. For you, also, I would say the same thing.
I forced myself out into the world, and I came to seem to function normal at times. That doesn't mean it isn't there underneath. I think that, if anyone has had real social anxiety when they were young, they will always have it, as a factor to be contended with.
Some years ago, I developed something new. I began getting pretty serious chest pain when I am extremely anxious about anything, social or otherwise. I had it two mornings ago, worrying about my finances.
I think people manifest anxiety in different ways. Anxiety can fuel all kinds of symptoms. I don't think that not having panic attacks means you don't have a serious anxiety disorder. My anxiety disorder lasted for many years into adulthood before causing the chest pain.
I think you, yourself, are the best judge of whether your anxiety in social situations is a big enough bother to say itthat rises to the level of a pervasive disorder. Do you plan things you do, sometimes, based around trying to avoid getting social situations to an extent that just is not normal? In cafeterias, I was always afraid that, if I went to join people to eat, they might not want me at their table. Now I force myself to join others, and it has gotten much easier. Still, the anxiety just comes through in other ways.
I think it is fear of rejection. Don't let it control you. Try and do things that seem a bit difficult. Sometimes, you will be hurt. Tell yourself that's not the end of the world when it happens. Good Luck.