I do that too. Sometimes I'll worry over going to just my therapist or pdoc. I love both of them, but I always have anxiety over seeing them. In the waiting room is torture for me. It seems like 5000 hours wait when it's only 5 minutes.
I've had to wait really long for appointments. I have changed doctors because of it. I'm not saying you should do that but if I need a doctor and they can't be available, I'll find one that is. I'm probably doing myself a disservice this way, but if enough people cancel enough appointments then some practices may look at the whole idea of "still accepting new patients" when they're overbooked. I'm probably wrong.
My 2¢ not worth one