You could be dealing with PTSD from your brother's treatment of you. I would suggest talking to a counselor to help you with your feelings. What you describe sounds a lot like flash backs of trauma.