Gosh I'm sorry for what you're going through.
I can totally see why you feel this way, and if this were me I would give him an ultimatum - me or her. I couldn't trust him with her after that, and in my opinion his marriage and family should be way more important. Ok, he says he doesn't want to lose a good friend, but hasn't he already done that by crossing that line? It's not a friendship any more and never will be. They have betrayed their partners and that's not right.
In my single life, I made a drunken mistake with a friend, it was stupid and should never have happened. We remained 'friends' but we never saw each other and only very occasionally said hello on Facebook. When I met my husband, I stopped this friendship altogether because my husband wasn't comfortable me being friends with him. I think if you love your other half, you should be prepared (as long as its reasonable, which it is in your case) to make sacrifices and if that means not maintaining a supposed friendship to save your marriage, then so be it.
Long story, but in a nutshell last year my husband started emailing this random woman whilst he was up doing night feeds for our newborn son. It started off a little flirtatious but went on to become extremely explicit and they'd exchanged naked pictures. I logged into his account, well actually a shared account (silly man) and obviously found all this out and went absolutely crackers. I told him he wasn't to have any more contact with her, he had to delete this account and also his own email account (they'd been there too) and for a long time when he went to collect his son I made him take the baby with him because she lived close to that area and they'd said about meeting up for sex. He did all this, without question and apologised profusely and still a year on puts up with my **** now. And I give him hell at times for it. I have forgiven him but that doesn't make the hurt and also the questions and paranoia go away, these things take time and trust is an incredibly hard thing to claw back once broken. Your husband has to understand this and take the necessary measures to get your lives on track again. After all, it was his mistake, not yours. I would ask him how would he feel/react if it were the other way round? What would he do? Would he be happy for you to remain friends? I highly doubt it. Stick to what you know you need, if he truly loves you he will do what it takes to earn your trust again.