Infidelity is the hardest thing a relationship ever has to cope with. Here is a website that you might find useful:

Steps to Recover from an affair/infidelity in marriage

The part of your story that bothers me is how much he would like to just move forward. That's easy for the perpetrator to say! Oh sure, it would be great to just move forward...for him!

This is going to take work. And that INCLUDES him. He should read that website too. It is fairly balanced. It doesn't give him the right to blame you for his affair, but it also sheds some light on how the temptation took a hold on him. Don't let him use this to blame you, though. He still gave in to the temptation instead of just communicating his needs with you better.