I often find myself too anxious to do something I'd planned. And once the doubts start creeping in then they often escalate until avoidance feels like the only safe option.

I wouldn't be too harsh on yourself, you're not pathetic at all. You wanted to go but were unable to because of anxiety, the same as if someone had a physical ailment like the flu or a broken leg. With you the thing that stopped you going wasn't visible but just as real. Perhaps you could tell someone in charge at the volunteer group how you feel and that you will try your best but sometimes may not be able to attend - they may even come up with some suggestions on how to make things feel safer.