Getting back later in the same day seems pretty normal particularly if you didn't specify this was urgent and to please reply as soon as possible. My therapist generally checked messages at lunch time and then at the end of the day, but might not call until end of day if I didn't specify a time or that it was an emergency. If it was an emergency I would have let his secretary know so she could be sure he got the message sooner and replied sooner.