Maybe this was all one big misunderstanding? From the outset, it sounds as though this guy didn't know how to respond to you at first - which could just be his general way with people - and when he realized the requirements for becoming a pastor he might have felt that his reactions to people and making friendships needed improvement. To that end he might have been trying to pick up where you left off only to be rebuffed because you've felt that he instantly disliked you and is now out to manipulate purely for his own gains.

I could be completely wrong but you don't go into much detail so i thought i'd explain how this struck me. You've spoken before about your issues with hyper vigilance and over analyzing. If the worst case scenario is true maybe it's better to play this guy at his own game should you ever happen upon him and his wife again - the worst thing for a manipulator is to know they too are being manipulated. So maybe shake his hand next time, smile, exchange a few pleasantries.

As for your wife, obviously she's entitled to her own friendships and opinions...so long as you can make excuses for declining involvement in any future arrangements between the three of them, i don't foresee any other problems. It's unfortunate that she can't better understand your side of things but conversely, if this guy is what you say he is, maybe you can take some comfort in knowing that out of all the people in this world...there probably aren't many like you who see him for what he really is. That's a sort of quiet power that requires sustenance from no one else but it still satisfying for the beholder.