I feel for you. And I have to respectfully disagree with Perna - it is a violation of confidentiality for your group therapist to identify your individual therapist. If someone follows you to your therapist's office, they are invading your privacy and they can leap to assumptions but they don't really know why you went into that office. This therapist has a professional duty to certain levels of conduct and he knows for sure who your therapist is.

In my group, I found out one of our group members had started seeing the group therapist individually. (she told me directly after a meeting.) It bothered me sometimes because it felt like they were continuing a conversation that the rest of us knew nothing about. But the group therapist never once said anything about doing individual therapy with this person or referenced anything directly from a private session. These relationships are very complicated.

I do agree with Perna that it is likely that the group members will think he was referring to a previous group session or things that have happened around group - unless he was very specific. But I'd definitely talk to him about it.