Call his attorney if he has one. I’m assuming you don’t have an attorney or you would be asking them for help.
Your temporary order should have a warning on it about not following the order. Take those steps. Tell him you’ll file for contempt if he doesn’t follow the plan. Tell him you’ll get makeup days for the days he kept them. File for contempt.
Make sure you’re following the order too. Be sure your home is not a detriment in any way to your kids.
You could have other options in your area, so reaching out to an attorney is a good move.
The court may have someone you can talk to.