You should get some advice from a lawyer about what you may need to do.

If he were to "take the kids," where would he be taking them to? This sounds like an empty bluff to me.