He is 'making' you, continue an affair?
If you are, uncertain about how adultery fits into, your religion, sitting with a priest, is a good idea. As is, speaking to someone about your ideation.
Your marriage, sounds, toxic.
Does it benefit children, to have two parents, who aren't interacting, in healthy ways? How can they model, a healthy relationship, in their future, when one isn't being modeled, for them?
Have you considered both individual and couples, therapies?

Sent from my LG-MS910 using Tapatalk 2