Sounds like you were picking up a bad vibe. Good for you. I appreciate how you got your husband involved right away. Follow your gut in this. My girlfriend has an expression for this..."You kept up your relationship guardrails"

If he approaches you again ask him his intentions on a phone call with your husband listening. Then see if he squirms or if you believe his answer. That may be the best medicine he has ever experienced. Take the high road. Cheers to integrity!