Can you tell him that he is hurting your feelings by doing that? If so, that would be my approach. Maybe you could write it down and give it to him if you are unable to say it out loud?
I'm sorry he is hurting your feelings. It would hurt mine too if someone did that to me.