Good luck. I had similar issues with my girlfriend and her kids, and eventually they got worked out. Trust is a good word to invoke here; if any of his adult kids can trump his prior commitments at any time, then his word isn't actually worth anything, is it? For the kid's part, they're not spoiled if he can afford them, and if they take no gracefully. I gather you've never seen that part but asking-not-demanding is a promising sign.

If the two of you are going forward, I'm glad but yes, I don't think that's behavior you should put up with.