I just read this entire thread. This is scary. I would be reluctant to confront this man face to face, even confronting him in the company of your husband may be dangerous. The man is not right. He may be released from a mental halfway facility and now homeless, camping out somewhere near the affluent homes in hopes of finding shelter. Or he may be a drug addict casing your neighborhood, your house among others. You just don't know.
I do know, from having been the victim of stalking, that people who are bold enough to walk right into your garage, several times, lack the social etiquette to be offended by a polite request to go away. This is a matter for the police.