Experiencing shame happens to remind us that we tried something and ended up experiencing the other person behave badly. Our emotions are there to help us so we avoid doing the same thing again that results in our experiencing something bad again.

Sometimes a family can be nice but that doesn't mean the person we are interacting with is nice. He is probably the sour grape in the bunch and it's simply not worth it to put any effort into him because he just ends things on that same sour note.