I agree with the suggestion you consider talking to him, perhaps in a while, when you feel ready and calm enough.

It's not necessarily cheating but it sounds like something you might want to agree boundaries on.

As a long time married person I'd say it's always worth giving each other time and space to be heard. I would approach this openly and without confrontation (that tends to make people defensive).