Tuff position to be in. Have you thought of getting a personal protective order. I dont know if you confronted him "legally" or "personally" but congrates on such a tuff action.
Remember, you cant control his actions or his behavior, nor are you responsible for it. He is only trying to "abuse" you AGAIN, doesnt matter if sexual, physical or emotional. Dont give him that option. It's ok to be angry with him. But remember if you show him that you are only giving him that right to affect you again. Stuffed animals are great for screaming at - they listen well and never talk back (be worried if they start talking back though hehe)
A therapist would be helpfull in this situation. Do you have one? And your always welcome to vent here, we all listen well.