Sounds like an unspoken agreement, you can complain, you can argue etc, he can not respond, or complain, so---in the end somehow you seem a "problem" and he just goes along doing his thing & expects it to continue as is.
If that is good enough, then it is good enough.