Though... I'm wondering whether this really is a counter-transference issue more than a transference issue.

Did he tell you precisely what it is that you do that makes him feel that way? Or was he just venting about how he was feeling?