Set boundaries. You have to.

If he can't respect your boundaries, or at least try to compromise or discuss it, then he's not putting in any of the effort: YOU'RE doing all the work, and that's not how relationships work.

I'd be prepared to leave if you had to. Good luck