Changing your life in any way, good or bad for any reason other than for your own satisfaction is not a good idea, nor does it stick.

Why do you find it hard to believe that he's such a heel? Why does this come as a surprise to you? No offense intended but the man knowingly carried on a relationship with a married woman. Is this perhaps his MO? Does he intentionally seek women out that are already committed elsewhere?

In my opinion if you truly want to get passed this, you really do have to stop thinking of things in terms of what he'd think of how you look or what you've done. What's past is past. Since this is not a work in progress relationship, why dwell on what HE'S done and focus on yourself.

What made you decide to seek a relationship outside of your marriage? Until you establish the real reason for that you're at risk of repeating it. If you were unhappy in your marriage, what kept you from doing things the right way (ie end your marriage, heal, begin a new healthier relationship)? What warning signs did you miss? Why did you tolerate this type of behavior? What were you looking for in this relationship? What trade offs did you make, why?

I don't mean that you have to answer to me or anyone else, you will do yourself the most benefit (imo) by using your energy finding the source of your unhappiness. In my view, enganing him in any form only continues this torment. You will never find complete closure here and seeking it is a fruitless effort. Focus instead on yourself and what you need to be happy.