Perhaps he has some charisma and good social skills and he knew what to say to get people hooked, that what often seems attractive.

However he lacks morals and would throw you under the bus any time, I am glad he isn’t your boss anymore.

And honestly usually (not always of course) if a man invites you out (lunch or dinner no matter), and not you and your spouse or you in a group, it’s a date, not a business meeting, especially if alcohol involved. So he thought he was on a date, hence suggesting going to his cottage. Yikes.

And who knows what’s really going on in his marriage, perhaps he isn’t treating his wife right or hits on other women too much. I don’t buy men’s stories about their bad marriage (unless I know them very well on a personal level). It’s often done to open possibilities of hitting on other women

I’d cut all ties. Not rudely but I’d just stop all contacts with him