I quite agree. Strictly speaking, you don't have to keep a journal and it's almost certainly to his benefit as well as yours that you have a chance to sort out what you want to tell him at any given time, and how you want to put it.

To his argument that he's entitled to know, there's the perfectly good counterargument that he's shown he often can't handle knowing. To his argument that he can't always trust you, there's the perfectly good counterargument that he's shown he can't always be trusted either.

It seems to me that this issue of trust and privacy is pretty central to your relationship and that unless both of you are working in good faith to resolve it, you barely even have a relationship.

I don't see where you have much to lose by making this clear to him. Of course there are different ways of making something clear, some make it clearer than others, and he's bound to be more willing to listen to (and "get") some, than others. Good luck in finding a way that works for both of you or at the very least, that leaves you satisfied that you've done your best.