That is frustrating and seems very unfair to you. You can try to talk to him about it again, but if he continues to do it and you experience it as damaging, I would consider finding a new therapist. I understand that seeing someone new can be difficult to think about, but if the alternative is feeling unsafe in your sessions, it might be the best thing for you to do.