Tell him you'll come see him, and that you're worried about him calling the police. Hopefully he'll reassure you that he won't.