It does sound like a double standard, plus he did protest too much.

His sounds more an emotional affair.

What you did, sounds more like a day dream, driven by dissatisfaction in marriage.

Sorry he's trying to use this, to his advantage, but without concrete proof of wrongdoing, most judges turn a deaf ear. Leave it to the basics, tight lipped, let him look a fool if he chooses to villify, publicly. Even if it burns inside.