Just to put out another way of looking at things -- you don't *have* to tell him why you want to break up. If doing so would be helpful to you, that's great, but if you want out of the relationship and struggling to tell him why means you have to stay longer than you want, maybe it would be better to get out, assuming that your mind is 100% made up and you see no resolution besides leaving.

Did you ever ask him what he thought about what you wrote? Or tell him that you were upset he never brought it up? If it turned out that this thing that happened two years ago was a huge misunderstanding, would you still want to end the relationship?