Do what you think is best, what is best isnt always good for the other person, and other times its not good for you. I think in your case, you should say something, when dealing with this behaviour.