I didn't get the sense that this was a near-rape...I didn't sense any violence, or that this is a person who will be inappropriate with any & everyone. It was directed at you. I do sense and understand the Hugely Wrong nature of this encounter. It sounds awful. And possibly drug or alcohol fueled. It happened. It was wrong. It seems there is something missing in the story, what led up to this situation? Perhaps you should speak to a crisis counselor to help you sort out what is best going forward to feel safe, or if you want to cut off this relationship all together. This is surely an option given your distance. Victims of incest have an especially difficult time reconciling this, and most have to continue to face their abuser up close & personal, as they are part of a family, but this is not you. There is certainly help available to you if you want help.

My previous post and point of view comes from one of compassion for you that you wished to have this brother be just that, your brother, & he violated that, which is more than heartbreaking. I am so sorry this happened. It is such a huge loss for you.
What do you hope to do? Do you want him in your life going forward with a crystal clear understanding of boundaries as many incest victims decide to do or do you wish to do as a previous poster suggested & expose this horrible incident to the whole (remaining) family as an assault? It is perfectly clear and very understandable if you choose this route. Only you can decide what to do here.