I wish he'd said, "Okay, but come anyway. It's important that we meet."
I'm not saying that you'd still go; you might have simply sent another text, "No, I'll see you next week" or whatever. BUT at least he would have let you know that he still wanted you to come, NO MATTER WHAT!
That said, it doesn't mean that he's given up, he just answered the wrong way. Maybe he was rushed. Maybe he was tired. Maybe he was overwhelmed . . . maybe anything. We always wish our therapists were perfect all the time, but they aren't, they are human. Go for your next appointment and let him know why his comment let you down. If he's worth his salt as a therapist, he'll agree that was a lame-o response and ask what might have been better!