If they call you manipulative again I would remind them that it's a condition of being human, and maybe add that a lack of empathy hasn't helped you any.

I know what it's like to feel alone. Try to distract yourself if possible, don't let your mind keep going to the dark place.