If I were to find myself in your situation, I would confront him, but not in an agressive manner. You seem to already think there is an admission of guilt. That is not a good way to go into the discussion. Talking to your T might be a good option to help get you cooled down.

I am in no way sticking up for him, but to ask if it was something he did, shows to me that he is worried about upsetting you. Not that he is admitting anything. None of us here know what other issues may be tied into this with the relationship or past experiences, but sincere honesty I feel is the way to go. Holding in these feelings of anger are going to do nothing for you, but rather make things worse when the next issue arises.

Take care, take time, and dont let emotions get the best of you.