I feel your pain. My ex-boy friend has sexual addiction issues and this made him so demanding of my time. I just couldn't take it. I ended up breaking up with him. He was also rather manipulative even with all the boundaries that I had given him.
I hope you and your husband work it out. Are you and him getting in help in RL? I hope you are.