Bottom line is you are both not the same as you were 15 years ago; one can't really "pick up where we left off" as one's experiences with each other and life have diverged so much. He scares you now; that's all you need to see/know; if you're not comfortable with him, you're not comfortable with him; there's no need to feel you have to remain in contact just because you knew him when you were younger. Friends are people you choose/like, not "forced" upon you like family.