Go go go
You may be running before you can be pushed or leaving before you can be left. Been there! I didn't actually not go but I wanted it. I did retreat. We did talk about not wanting to come but otherwise kept it very light.
Or you may be punishing him for not agreeing to email on holiday- which is only really punishing yourself. He's not gonna call and beg you to come to session. Been there too.
That distance thing - do you experience in life that when someone is not there they are effectively gone/don't exist/dead to you? That may be what you are feeling with your t after a few days. Just another manifestation of the insecure attachment gig.