You feel what you feel about. I can see it both ways maybe depending on how you two generally interact and depending on the context and tone of voice. I can totally see my T saying something like that in jest, a bit of kidding around. Just today he gave me permission to take off my shoe and throw it at him but he warned me he'd throw it back at me. He refers to the hospital as the (my pdoc's name) hotel. That kind of banter is normal for us, so if he said what your T said it probably wouldn't phase me.

However, if it was said more seriously by a T that I didn't feel that comfortable with, I might be really bothered by that remark.

So you'll have to think about it in context (something none of us here can do) to decide how it was intended and go from there.