You are deciding what he is thinking by looking at him walk by.
The only way to know what he thinks is to talk to him.

I once had a co-worker that I thought hated me. This idea came from the fact that when we passed he was always frowning.
I was in REBT (Rational Emotive Behavioral Therapy) at the time and so I challenged my interpretation because I had no "proof". To get the "proof", I decided to just say "Hello" to him a few times and see how that went. And I also talked with another co-worker about this guy, just in general wondering 'what is he like?'.

What I discovered was that he was not only working full time but also going to school and close to graduating and getting his degree. He really wanted to stay on and move up in the company so he was working very hard. He was also part of a team that was developing a new product for the company.

As you can imagine, this was quite a load for this young man.

The first time I said "Hi, XXX!", he looked at me as if he was just seeing me. The frown melted into a big smile. I continued and, although he had little time for small talk, we got to the point of exchanging a few words.

From then on, when he went by frowning, I knew he was merely thinking as he walked, maybe his mind was on the task he just finished, the one he was returning to, or whatever.

I have since tried to not interpret facial expressions, and instead go for the "proof".

So the next time he walks by, smile and say "Hello!" and see where it gets you