Your littles are telling you something... Probably he has shifty reasons for the commuications...My Ex did And I was where you are at... Really.
Protect yourselves....Limit the calls...make he knows he is not the center of yr world w/out threatening him. Do not open up too much to him and make sure your place is secure.
If it's any consolation, I don't blame you for being scared.
Please Take Care