This actually happened to me yesterday, during a minor "heated discussion" with my son. It bothered me Overmuch. I know, and didn't say anything, that the reason it bothered me was because it was used as a phrase to dismiss me/my response. {...and a little while later, he knocked on my door with an offer of a potato pancake for breakfast

} So, yeah sometimes it bothers me, depends on person and situation.