I would suggest couceling. I've lived with this before. But here is something you may not have thought about. People tend to see in other, and accuse others when they themselves have that issue. I hate putting it out there like that. But when my ex- was accusing me, come to find out, he had been cheating himself. I don't know you or your husband, so I can't be a judge, but it's something to think about.
Definitley try counceling!
JJ