Whew!! ((((BP))))
That's VERY scary in my opinion. I hope you'll inform someone that there is a strange man lurking around the neighborhood. It couldn't hurt.
I'm like that about my anxiety too, though. I freak out sometimes when I shouldn't. But after reading about the man around the house, I'd definitely notify police that there is someone there that looks suspicious. The least they could do it circle the neighborhood a few times a night (and\or day).
Be careful whatever you do!! I'm glad hubby is going to be there with you for a few days.