How much time apart was there between your now current but then ex and the 2 guys? That might be the reason he's having trouble with it and I agree it still about self esteem. I mean, if it was only a week or a couple days, he may feel he wasn't important enough to grieve over the loss of the relationship...you were ready to move on immediately. And sorry, anything shorter than a month is 'immediately' to a guy. Even then, it would be 'too soon'...unless of course, he was doing it. (equal opportunity: women feel the same way about ex's...I do at least)

It's still about his insecurity of course, so yes, be understanding. But at the end of the day, you two were NOT TOGETHER; however, while that tastes like justification at first bite...you'll find actually swallowing it to be a lot tougher. I agree..tell him about the second guy. And absolutely remind him, you chose him...do NOT say you feel guilty or dirty. It happened because he was not with you. Period. Too much explanation sounds like guilt and with that little brick he'll build an entire city of jealousy and supposition.

After you've told him, you might suggest couples counselling if you still feel he's having trouble...sounds like getting back together didn't fix the original issue...at any rate...couldn't hurt