One approach would be to set time boundaries. For example, you will not speak after, say, 9:00 p.m. your time. You will not speak for more that X hours per day. You will try to be prompt but you do not promise to arrive at the exact minute that was agreed. You would tell him these boundaries.

If he gets anxious or scared, that is a problem for him to deal with.