He was caught fooling around so you owe him nothing, he should be the one to try to make things right but clearly he has no intention of doing so. He may have feelings for the other woman that he is in denial about and causing him to become closed off from you. If there are no signs of improvement I say it's time to move on. It's not easy but for your own sanity and well being it's a must. Good luck