Are you sure your sessions are supposed to be as long as you think? Depending on the billing code, they could be 45 minute sessions. It is a different billing code than for longer sessions. You might ask.

I doubt this is about him not wanting to work with you. I doubt this is about caring less for you than other clients. I haven't found most therapists to act quite that passive-aggressively.

Clarify the number of minutes your sessions are supposed to be. That is the place to start. He may schedule every hour, but bill for 45 minutes so he has time between clients. Explain how starting late is causing you distress and see what he has to say.

The CPT manual provides for flexibility by identifying time ranges in the descriptions of the three codes, as follows:
90832: 16 to 37 minutes
90834: 38 to 52 minutes
90837: 53 minutes or longer