A couple of things that work for me:

-- If I notice that I always seem to be doing things for them, or paying for things, and what they do in return doesn't come close to evening the contribution (they are unduly feeding off my good will)
-- If I feel badly about myself around them for no good reason (they enjoy being intimidating, feeling superior, etc.)
-- If I hear from other people that they reveal confidences I've shared with them, or that they speak disparagingly about me (they are two-faced, don't respect me)
-- If I notice that they focus on the details of my life in an unusual way (could be cataloguing information about me for use later in a negative way, to terrorize me, stalk me, etc.)
-- If they generally can't resist an opportunity to be critical or negative. While this could be signs of many things, some of which might be understandable or forgivable, for me I can't be around that kind of energy for extended periods of time -- toxic for me.