I think i would believe your daughter. Hasn't he been proven to be a liar? There is a much higher % of Mothers being awarded custody than Fathers, so I would keep in mind that for several reasons you have a better chance of getting the children than he does.

Of course, I didnt know you when u were going thru the divorce, but you clearly state you are stronger now than you were then , when he bullied you during the divorce. And I don't really know you now, but I do know that you love your children, and want what is best for them, and from reading that horrendous letter from your EX, I think most of this is just an ego trip for him, another way of being controlling. The mediator will certainly have input, and hopefully he/she saw his "pftttttt" with u attitude !

Try and stay positive, do everything u can with your kids, and take him up on his written offer of you getting the children even when it is his week ! Its in writing !! Best of luck to you and the children, and it is very clear you have lots of support here !