Yours is a complicated story. I would consider marriage counseling. He is going to have to earn back your trust and respect. It's going to be a rough road but if the both of you are committed it could work. I would be wary of his continued contact with his ex unless it is about the children.