That does seem a little bit complicated, as we can project our fears onto our T's at times. But I would tend to trust my instincts if it were me. I have often ignored someone's disrespectful behavior toward myself and then months or years later had it suddenly pop into my mind again, demanding to be addressed because it really wasn't okay.

Perhaps you could consult with another T about this, either before returning to therapy with him or a couple of times just after you restart therapy with him? Usually the standard advice is "Try discussing this with your T," but I think this is something it would be easy for your previous T to deny / be in denial about.