I assume he wouldn't go into counseling with you, so this topic is another one to discuss with your therapist. I think I would leave him, but maybe that's just me.