If it happens again..call the police. They will make him leave (in the USA). You can also get a protective order so he is not allowed to come near you (again, in the USA). Bless you and your children. You will be in my thoughts and prayers. I have a very dear friend at work that is going through the same thing...

Be safe,
Susan