Don't be friends with him if that idea makes you cringe. You are doing the right thing to stay away from him.

He might feel badly about your decision, but stringing him along out of concern for his feelings, or pity, would not be doing him any favors. Be genuine: follow and express and live your truth.