I have this issue as well, and yeah, it makes anxiety worse in my experience as well, especially that I already have issues of staying present in therapy for example and not get disconnected from myself. And then there is guilt because of the being late.
I also have been having this issue at work, since my issues started with depression (though I wasn't aware at the beginning). I basically couldn't get out of the bed, it took me sometimes two hours.
I think it's related with anxiety, so anxiety makes me procrastinate, then I realise it's late, then there is more anxiety.
Maybe the only solution is to really be conscious about the issue and not try to distract yourself that there is still time. I managed today to be on time, making a pact with myself that at 10:20 I will start preparing to leave, and so I did.