Life happens and brings changes. Your PDoc was reaching out to you because he cared. He knew you would one day move on. The point is that he wanted to help change your life for the better - while he could be a part of it.

Don't burn any bridges. Just be very honest. Let him know the facts. And let him know how you appriciate him and how you will miss him.