I have depression (take meds for it) and social anxiety. I have three close female friends, and, I should add, it took me many years to build these friendships. I say NO a lot to invitations for outings. Sometimes I don't even answer the phone when they call. But, they are true friends and give me my space. A real friend should understand and give you the space you require without getting impatient with you. You have every right to say no when you don't feel like joining in.