I don't know exactly why you had such a strong reaction, but you are probably right in that part of it is a feeling that he doesn't want to be close to you. Could you ask him to move closer to you so that you can still feel connected but he can face you? My guess is that he wants to face you so that he can see your body language. Body language tells a lot about how a person is feeling and if their words are matching what their body is saying.

I had a session where my T ended up leaning forward in her chair. She was trying to make me feel connected to and heard, but for me it invaded my need for personal space (even though she was across the room from me). We had to talk about it the next time because I felt so invaded even though she was trying to connect.

So it is totally normal for you to be sensitive to where your T sits. Talk about it more.