If your delivery consists of being "mad," then his defenses will go up. He calls you controlling, yet you claim he gives them a head's up before he does anything, and they constantly phone him.

Ask him, without being irritated, or coming across as though there's ill-will behind the question, what he thinks of the relationship, and if he is okay with it. If you already know the answer, don't bother. If he does not see this as a problem, and is fine with codependency, then you know what to do.