Terrible person? probably not, from what you describe he's the one that provokes. Another question is does he still act this way towards you? and how old are you both? was this all recent or well in the past?

On the other hand forgiveness is a way of letting go (of the pain), and frees your heart and mind to be clearer in looking at these things, your family, and yourself.

As for feeling wronged and wishing some one would have had your back, understand that even in the family dynamic there are times when you have to stand up for yourself. If you're not a big guy and not into slapping him in his place, you need to at least get him to know how much damage he's done and not carry it around secretly. If your brother really loves you, he'll eventually understand, even if he doesn't take it well at first.

But carrying around resentment deep inside isn't healthy and I hope you find a way to let go, and have a healthy relationship with your fam.

Best wishes