I can't speak for OP specifically, but for a lot of people this kind of situation reinforces a negative self-perception and can also touch on abandonment and other issues. I don't think it's as simple as one action is right and the other is wrong. Especially in the unique space of therapy, where the hour is usually completely about the client. That's one of the most healing things for many people in therapy, so to have that disrupted can be hurtful.