I am in a similar position. People that are highly N trigger me like crazy. I just learned about it / discovered / identified it, though, this year (put a real word to the behavior), so I am feeling a little bit better now in my dealings with them, mostly my family members (I avoid them like the plague ).

I still don't know how to deal with Ns at the office. Yet. It's hit or miss while I work on developing a thicker skin.