Even if anxiety is not quite the right word to describe it, I think you will find other people with similar feelings in the anxiety forum. I don't have any physical issues either and public speaking doesn't phase me, but sometimes group events where I may encounter a lot of strangers, like parties, make me uncomfortable.

For me, repetitively going to these sorts of events and interacting with people even when I don't want to has made it easier to do. I still don't like doing it, but it is easier. It sounds like you have tried this but it is not making a difference? What would happen if you made yourself go into that room during break and greet a person? What if you did it twice a week for a month? Have you tried that already?