I would say be upfront and talk to him about it and tell him it bothers you. If it's making you uncomfortable, it isn't something he should be doing, especially behind your back.