I think it's something to bring up. Was there other insight he had that he would have based that comment on (not just the T thing, but other areas of life? relationships, work, hobbies, etc?).
T's tend to make some assumptions, though if he based it on just the T situation, it's a pretty huge assumption to make...
Like A Red Panda, i've learned to trust my T by trial and error, by pointing out times she is just wrong about something, and by recognizing when she's right. For me, therapy is a learning process. It's T learning about me, and me learning about T and myself.
If I recall correctly though, these aren't the first inklings of doubt you have had about this T. Does looking at the overall relationship you guys have warrant a change in T's at this point? Or is this something you would like to work out with T?