My therapist is never brutal. It's also part of his way of working that he does not regard himself as any sort of authority about my life. I would feel very uncomfortable with a therapist who was argumentative and made guarantees about the behavior of other people. I also wouldn't welcome a therapist "challenging" me. It sounds way too authoritarian for me.

Your situation sounds hard - he sounds defensive and difficult, but I know it can be very hard to move on. Though it's sort of strange that he seems to be suggesting you stayed in a previous bad situation longer than you should have in the past, and yet appears to be suggesting to you that your current problems are all your fault and getting out of your situation with him won't help you. It sounds like he's discouraging you from listening to the wisdom you have within yourself.