I tend to overthink so I can understand this. It sounds like he's unsettled you somehow.

If he was hitting on you it sounds like H put him in his place. If it helps any I'm fairly sure a friend of my husband tried to hit on me once, it was confusing but I had zero interest, pretty soon afterwards he had an affair and left his wife, and then another affair and left that woman. So I've always felt my instincts were bang on - we normally know when someone is hitting on us even if we'd prefer to think they weren't (if that makes sense?).

If a person gets an emphatic 'No' it usually puts them off. It did with my H's friend and it sounds like it has with yours too.