You are not alone in this except for I did not know he was going to step out or end our session early. I was in the middle of something and he got a call that he needed to take and then ended really abruptly. I was so upset and felt "dismissed". I did the same thing in cancelling and then changed my mind. When I went in there to explain he did not even get it.
I never get frustrated about stuff like this and he knows that. It was just this one time b/c something else was going on and like you I remember a parent doing that to me all the time.
Here is what I finally realized for myself. It's not that he did not want to understand it was that he got too caught up in his world to. You are not the worst patient and don't put that on yourself, I get that you have enough going on without blaming yourself for his lack of seeing the whole picture. If you T is good and you like working with him then stay with this and decide if it is worth looking at for your own well being.