So am I correct in understanding that it's not a germ/ickyness issue, but anxiety about the situations you might get into?

If possible, I'd suggest not waiting until you really need to go. That will limit your options, place you under a time constraint, and add to your stress overall.

People sometimes forget to throw the lock/occupied sign or fail to do so correctly. So maybe you want to knock before entering a stall or single occupant toilet?

I don't know if this helps ... I have to clean washrooms at work. I'm a guy. So before entering the women's washrooms, I knock loudly and call out "cleaning service" then wait a few moments to give anyone a chance to respond. Obviously I wait if someone is in there and can leave if someone needs to use the facilities while I'm already in there. And for what it's worth, I have social anxiety. (Gee, I didn't think I'd be typing *this* message when I got up this morning. )