I would just encourage you to not swallow your anger as somehow "misplaced" or "too much" or "inappropriate" or "unfair." I also think it would be very difficult to not hold onto residual guilt if you walk away. You deserve the confrontation with him, he should openly hear and accept your anger at his response, and you both should get the opportunity to see if there's any benefit to continuing. But like ruh roh I, too, feel a sense that he's currently seeing this as a "learning opportunity" for you, rather than as a mandate for his own attitude adjustment. And as Oliviab acknowledged, it's not your job (and I'm not sure it would be possible for any client) to help him confront this in himself.