I don't think it would hurt to tell him your annoyance, but I can't see any single thing he did wrong. He had discussed boundaries with you before your text. He called on his day off. He "hoped" you were okay--to which you could have replied "Not"-- and he cleared his schedule to meet with you the very next day. And you say the crisis had been averted by the time he called. As they say on the phone: "If this is an emergency, hang up and dial 911."