The children could be resentfull that you are divorcing dad as you did so much with them they feel betrayed and miss you already, I know that my husbands ex wife and her children who he looked after for over six years (their dad had died so he was dad) was very resentfull towards him for leaving as he had done so much with them.

I think what I am saying is it wouldnt take much for 'dad' to turn them againts you but it wouldnt take much for you to get the love back if/when he calms down and lets you see them.

I am rambling I hope some of that made sence.