Is he still having affairs?
Do you know why he had the affairs to begin with?
It sounds like he may be or have been using sex to medicate himself... make himself happy.

I would say it all depends on him. It sounds like you want to try.
If he has no desire, then nothing will change.
If he has been faithful since and really want to be with you, then yes, do go.