He's already cheated, you have reason to believe that he might do it again and he even told you that he might leave. And now you are still trying to work it out for him. Meanwhile he has done no such thing for you.

Your equivalent of holding things down is changing your decision on sex and maybe having more sex with him.

His equivalent is not leaving you.

There is a major mismatch here and it looks like you're doing all the work.