I have noticed much of my anxiety socially is around people I don't know well and people I feel judge me. For example, I have neighbors that I only met once and I have a real fear of having to interact with them. Another neighbor I have known for years and I don't mind seeing them. In my family, I have a brother that I feel judges me. I have some of that same anxiety with him and dread seeing him.
It is as if I label people as 'safe' and 'unsafe'. Socially, I can interact with the 'safe' people and even enjoy it. I try to avoid the 'unsafe' people and am filled with anxiety when I do interact with them. Not knowing someone isn't a reason to feel anxiety with them, is it? I could understand feeling that way with someone that I feel judges me. There is a distrust.