There is no way to go around it other than to just ask him why it's there, and if there's something affecting him. I'll be honest, If I were in your shoes I would feel uncomfortable, because I would try and find out the 'why's', but maybe there's something to understand that's there once he tells you his reasons.

My partner once would always mention this one girl in his past, they were best friends who dated and that ruin their friendship. He regrets the way they stopped talking, they just drifted apart, and after the years he wanted closure. That was understandable to me. They finally contact each other online and via phone, showed me the convo and that was the end of it from there. Now he felt relief because he 'properly' ended that chapter with her.

Maybe, your partner needs to end that chapter with her for good, maybe there's something there he never asked/said to her or if he's still holding on to it mentally? That doesn't necessarily means to throw the letter away, but like you said keeping it on top of his drawer is sort of like a constant reminder there not only for him to see, but for you as well. Maybe he needs some sort of closure but doesn't know how.