It is unfair. They can't talk/discuss/yell at him for some reason and you end up the scapegoat/only one left; the classic "kick the dog" scenario. See if you can feel sorry for them, not take it so personally and then it won't hurt so much. They are frightened and frustrated and taking it out on you but it is not about you so just try to smile and say, "I'm so sorry" over and over (freak them out by being kind :-)