You said your marriage has been good most of the time. So, I think there is no need to amplify the issue if he didn't actually talk to anyone. Instead of thinking of breaking the trust with him, ask him why he was fooling around? Does he think your marriage could be better. Work on that. Mistakes don't just happen from nowhere, usually. This is my opinion.