I think that you need to look into family therapy. You and him need to be on the same page with the same age appropriate expectations for all of the children or it will build resentment in the kids. Good luck.