Wow, that would bother me. My T usually come forward. I would just see what happens next time and if he does it again and it bothers you....say something. Ask him if there is a reason he moves his chair back from you. Of course, this will probably spark a whole, "why does this bother you?" response, but it might be something to talk about?!

Good luck