Maybe couples counseling could be helpful? would he even consider going? Regardless if he will go to Therapy or not, You should find a therapist to help you manage through this situation and find out what is going to be the best thing for you.

Personally this would most likely be a deal breaker for me.

I'm sorry I hope you can find some hope.