That's an odd thing for a friend to say. I don't know if he has any genuine grievances against you in this respect, but it's still a bit insensitive to make this attack when you've laid out your private thoughts for all to see.

I assume your 'diary' is also for your own cathartic purposes, and not simply to inform your friends? If this is the case you can probably tell your friend that he doesn't have to subscribe to your account.

Tell him that he can compare your entries in the written journal against each other to find inconsistencies. This will require that you let the journal tell the whole story and don't add to it 'offline', so all the facts will be there in black and white. This should at least end the debate as to whether your story changes.