In my opinion, neither of them is a good choice. Abuse, in my opinion, is a deal-breaker and you deserve to be safe. Adultery is also a deal-breaker.

You may have to maintain contact with your ex-spouse for the sake of your son, but you don't have to live with anyone.