I think you are seeing it correctly for what it is. He's bored without his wife there, so he's reaching out to you. He would cheat with you, if you let him.

I'm of the mindset to not burn bridges, so I just wouldn't encourage him by flirting back, but keep the occasional contact. He and his wife may be able to help you in some way.