He is ignoring you because he knows on some level that it bothers you, and that the more he ignores you the more likely you are to continue the cycle you're in with him.

Ignoring someone like he's ignoring you is a great way to drive them nuts, speaking as a narcissist here. If I want to control someone, if I pretend they don't exist they're inevitably going to start wondering what they did wrong and why I'm suddenly acting so cold and seemingly controlled to the point of being clinical around them and no one else that they are aware of. They'll also wonder how to get me to stop ignoring them, if they're someone I've known for awhile.

Intermittent levels of attention and indifference is an extremely effective method of control that's also quite covert.