Maybe you could pull him aside over the holidays and simply talk to him and explain how you're feeling.