I don't think you mishandled it at all.
He wasn't thinking. Seven minutes is not enough time. My guess is he had an appointment until 1:50 and left himself a few minutes wiggle room to wrap it up without realizing that took up all of your wiggle room.

Hugs, I wish you didn't have to worry about reasonable requests/actions being perceived negatively. It is reasonable to say that time won't work, can we try after 5? or to not answer the phone if you're genuinely not available. Those are actions that should be perceived as good boundaries.

It sounds like you both need some clarification about timing and how to check in and what will work for you.

And I do wish you had someone in your corner that you feel is a better match.