first thing ....it is never too late to report rape!
second, now your mum has backed you up against your brother, try talking to her, explain how the hurtful comments he makes affect you and how you do not want to be around him any longer, let her know you do not want her to choose between you, she can't, but that it would be best for you if she let you know when they are coming so you can make other arrangements so your path does not cross your brothers.

third...if they arrive unannounced, you have every right to walk out of the room and go to another part of the house or out for a walk. you do not have to stay in the room where he is just because he is there.

fourth... no one has the right to put you down or make you feel how they do...you are better than them, remember you are a survivor, that is something he will never be!

be strong and focus on the day you will be able to stand on your own two feet and move out. x