Your ex sounds like like MY ex, except for the strong physical attraction. That flew the coop after he started acting weird. And I never really felt that attracted to him anyway. My parents basically forced me to marry this guy -- I was 18 and his family was well =-t0= do, and my parents I guess felt I couldn't do any better.
He drove me almost insane. I spent 2 weeks in a sanitarium. That helped, but I continued with therapy for years after I got out. I also divorced the idiot years after he started doing this stuff --- I waited a long time because he threatened me that he'd kill me if I ever left him, and he said he'd take my kids too (if he didn't kill me of course). i believed him because he WAS capable of doing that.
Your husband needs some help but I'm sure he won't agree to any. Mine didnt. He felt I was the one who needed help.
I'd stay away from him -- and you should get therapy too if you haven't already. You need to find out why you're still attracted to this guy -- I think it goes deeper than the physical. Best of luck and God bless. Hugs, Lee