I think that it's fair to be on guard and fair to want to limit your travels to be there. There's plenty of stress in life without adding to it, more than you can handle or desire to handle. But this picking up the phone to listen in to the conversations that his wife is having is alarming.
Expecting pricey dinners out at your expense as opposed to being appreciative of a home cooked meal...well, that's a lot to demand of others. Negates the nature of gifting and wheels into the land of entitlement.
Add in the alcohol and considering my own life's perspective, I say be there but at your pace. If she's an enabler, don't be hers, if that makes sense?
I'm sure you'll in time figure out what to say and how you want to say it if there's really much to be said. Boundaries are a curious thing because they aren't necessarily about the other person.