I would contact a lawyer if I was you but don't tell him. Unfortunately, you have to consider that he may have already cheated and this is his way of trying to get your permission after-the-fact.

Soooo sorry. He's right about one thing; you do deserve better.