Seems an intense reaction, many people would not even notice and have no trouble overlooking him. If I were in your shoes I would be curious as to why I felt so strongly about this guy. Someone else just "existing" shouldn't make you angry. Maybe he reminds you of someone else?

I would laugh at myself for getting so angry/uncomfortable and shrug. That you are afraid you will blow up at him says, to me, that you are paying too much attention to him, and that would be your doing rather than his? He "fascinates" you in some way, is a lightning rod of some sort for you?