You are NOT a loser....
You are not alone with staring at a certain spot....I do the same. When I'm uncomfortable or whatnot, I usually stare at a certain spot as well. One day, I moved to a different seat to try to work on my posture issues that are contributing to my neck issues, and the one thing that threw me off was not having THAT SPOT to look at. So, know you are not alone.
And I completely understand your feelings about having your session disrupted and T not even knowing where you were. Even though that happens to most of us and we don't think much of it, it's very, very difficult to not take it personally when it comes to our Ts.
I hope you are able to address some of this with your T.