Have you thought of asking him if you have upset him in some way ?

Perhaps you can say .....

Have I upset you in some way only thats the impression I get with the way you look at me ? Maybe I am being paranoid if so I am sorry.

Or

Smile at him everytime you see him see if that works ?