Sometimes people say things and they don't understand how it sounds. He might be blaming himself for your problems, but it comes out like him sounding like a jerk.

I know this is hard to accept. It took me YEARS to accept that my parents don't understand what it's like to live with bipolar disorder. Hang in there.