My experience was much the same, tho my parents were divorced and us kids lived with our dad. I was always the "bad" one, tho I never did understand what I did that was so bad. My sister was the one who did no wrong. I'm in my 50s now and I have tried to put the past behind me. Even tho my parents harmed me, I still love them. And my sister and I now have a good relationship.

My advice to you would be if you can put distance between you and your parents/brother, do it as much as you are comfortable with. Don't feel guilty about it as your first priority is to take care of you. Things will probably change as time goes on, but always ALWAYS put your mental and physical health first.