If it makes matters worse, then I definitely wouldn't try to argue with him, even if his reason sounds selfish. Maybe she only wants to spend time with her father and doesn't want to go anywhere??

Sorry.
I hope you can find some sort of counseling for your daughter. It doesn't sound like you'll be able to reason with her father.