I think this is so hard. And when it continues it can make you stop saying yes at all, because you start to fear that you will not feel like doing what you agreed to, when the time comes.
Maybe try to identify what's behind the anxiety about this, with your therapist?