If he calls, you could just suggest you grab a coffee. If he says yes, get there early, already seated, so you have the choice not to get up and great him with a hug. Should set the stage for you to explain, or even get the message across all by itself.

Not the most elegant way, but that's how I would do it if I wanted to avoid a confrontation.