Yes I can imagine it may freak out a man, especially a younger one, not because he doesn't like or love you but because he is not ready for that phase of his life. He has other things he wants to do and achieve first.

It can happen the other way around too, I know a young woman who was freaked out by early baby/marriage talk by a boyfriend, this among other things lead her to finish with him. Overall if people have different life aims/plans it's best they find out though.

But reading your diary without permission is not good. As avlady points out it's a sign of lack of respect which may well mean he will disrespect you in other ways too.