You don’t come across as offensive (I was diagnosed as being autistic, but high functioning). It is a spectrum and, unfortunately, the reality is some autistic people can be potentially unsafe to be around (obviously, not because they want to be). It’s ok to acknowledge that you feel scared about visiting and to be thinking if there’s anything you can do if something happens. I know this doesn’t really help you, but I hope the visit does go ok.