Most professors have posted office hours. It might be good to talk to him then, alone, where you won't be interrupted. It may be that he was just referring to book, or that he doesn't believe the stories he has been told. He may not be connecting this with you personally at all - just because you were in one doesn't mean you were the patient.

I say this because my husband was a tenured professor for years before going into administration. I could see him saying something like that and have no clue that you had a note from a T in the past, remembering nothing of it.

I hope the professor realizes what he said and how it was taken and apologies.