If he goes home after your stop, i would ask something leading like, "does your wife have lunch waiting for you when you get home?" Think of a few lines like that beforehand, and the right one will be ready when the time is right. Thats how stand up comics practice!