TSG has it about how I'd handle it too.
If he took you out for dinner, he is going to pay for it. If he asked you along, you pay your share. If he asked you to buy, that's all on you