Inky, tell him no. There's no reason for any respectable person to insist on having someone come to their home, especially when it's a situation such as you have. Tell him you'd be happy to join him at a park for the annual Police BBQ or something instead.