I don't think it's necessarily "bad" that you reacted the way you did. That kind of thing would have grated on my nerves too. And what you said to him doesn't sound inaccurate, from the way you described his behavior.