I think that if he questions you, you could simply say "I have anxiety". If you want to introduce this part of yourself without him asking, you could say "I'm sorry I haven't managed to get away before, I have anxiety"

I find this to be a relatively "safe" thing to say to people I don't know. It has such broad meaning. It doesn't scare people away, it explains a lot of things, it doesn't require further explanation....

I hope you get to go and go soon. Could you go when you're sister isn't around so you don't have to tell her why you're out?