I agree with the above and suggest you threaten him with the police, since there was that incident with physical violence. The police would not protect you from being tormented emotionally, but they will act on a report of domestic violence, so it would not be an empty threat to give him clear consequences - one more incident and the police will promptly come to your house.