Lion

I think you and your wife have to approach this man directly and honestly. Stop playing mind games with him as he seems to enjoy them. Refuse his gifts to either you or your wife. Set boundaries with him on what is acceptable and what is NOT acceptable.

If he refuses to obey your boundaries and continues to act out, distance yourself (and your wife) from him. Make it very very clear that he has crossed the line and it won't be tolerated anymore.