This man is controlling and abusive. My advice is to end this relationship at once. Under no circumstances should you move your children in with him. Once he has you and your children under his daily control, he will become even more controlling and abusive.