I hate that too, I really don't like people but especially men in my space. It's like your brain knows you're perfectly safe but you're still freaking out. it's hard and I'm sorry that you deal with it.

A guy I know in passing came up and put his arm around me while I was at work one day and I get edgy any time I have to be in that office.

Yeah I think it's totally okay to ask people to give you some space. You could maybe claim you're claustrophobic if you don't want to have to say the real reason.