I think it's ok if you need to distance. You don't have to get into the why.

I think it's thoughtful of you to not want to hurt his feelings. Yet, you want to find someone nice like this that is ready to have a relationship with you.