Oh my! Definitely do NOT have anything to do with this man outside of work. Now, I don't mean to sound too harsh and I admit I am biased because I am trying to salvage a marriage that has been interrupted by my husands infidelity, but this whole thing just smells bad.
Any man who speaks ill of his wife is a schmuck. Any man who tries to discuss his personal and sexual affairs with his wife with a woman he works with deserves to have his head flushed in the toilet.
Seriously, ignore this man and his inappropriate attempted conversations with you or else you are going to get hurt. And after you get hurt you will have to deal with the guilt of playing a part in hurting his wife.
At lease you are smart enough to see that this is already a problem. You must stay strong and stay away from the married men, especially this one.