First of all, i cannot imagine how scary and violating that must be to have seen the man in your home unnanounced.

i don't know where you live but where i live they need to post at least 24 hr notice of entry, and add something like, 'between the hours of 1-4' or whatever it will be, so it's not an exact time, but you know the general timeline.

I realize this is extreme, but is it possible for you to move to a different place where they don't come in so often, and so disrespectfully? I normally wouldn't suggest something so life-changing as a move, but considering your dx, the way they go about things, and the trauma it's causing you, it may be worth it for peace of mind, kwim? Because being in a constant state of anxiety over 'what-where-when' is no good for you at all!

If that's not possible, (and perhaps something to be done before looking into a new home) i second what someone else mentioned, that perhaps talking to someone in management and explaining your situation may help them narrow it down a bit so you can arrange to have someone with you during these inspections.