You're not wrong for being annoyed, just reiterate to him that you're involved with someone and maybe ask him to please not say things like that to you.

If it makes you uncomfortable, ask him to stop. Laughing it off will just leave him to say something like it again some time in the future because he'll think it's okay.

Edit: Reading all of your replies, I really don't think it's necessary to hit him at all. Sure, it might solve the problem, but I don't think it's right to hit anyone.