I don't think you have to tell him if you don't want to. However, since one of the things you want to work on in therapy is being able to connect with people and yet when you did feel that connection with him you also felt pain, it might be helpful to you to talk to him about it. It could help you understand yourself better.