If he was kicked out of town when he raped a 12 year old, I'd go tell the police that (rather than having your friend tell the police he was in a fight with a guy the night before) and see if they can't do something about that. I don't think he should be around?

I certainly wouldn't go anywhere alone and I'd maybe take a self-defense class like Sky suggests. I don't understand why you're hanging around with such toughs at your age though. I didn't know anyone like that who would show up at the kind of parties I went to at your age. Run with a nicer crowd or home earlier and he's not likely to run into you.