My brother did it to me too. Only my sister knows, I haven't have the courage to tell my parents because my father is like yours and so is my mother.
Hung in there. It is ok to not feel anything beautiful for your brother, just don't let hate enter your heart. Let the good things fill your life... and just focus on them.
Ask for help if you feel you need it, and keep talking about it with us, or whomever you feel comfortable. Talking helps heal.

gab