I think you have to tell him your concerns, that you can't entertain every night and that you think his drinking is causing a problem for him. He doesn't work or do anything during the day? Maybe you can plan a weekly, innocent, activity like grocery shopping or going to the library or to a coffee shop, etc. that happens early enough in the day that he won't have been drinking yet?

I wouldn't go with him in the car at all if he's been drinking/you smell alcohol on his breath and I would tell him why you won't get in the car (matter-of-fact way, not angry or anything). If you don't value your life, why should he (value either your life OR his?). A lot of help could come from how you represent yourself; not being a doormat for him to come to each night (don't let him him, explain you're "busy" and/or throw him out; it's your home, he can't be allowed to stay and wreck your health/keep you up, etc.).