I think you used excellent judgement and made an intelligent choice.
You may have also felt fear, but that's understandable!

When a person makes fun of another, they are saying something about themselves.
If I make fun of my neighbor because they drive an imported car.... that is about my feelings about imported cars and people who choose to drive them. It isn't about the person who drives the car of their choice; we all drive the car of our choice.

I agree that in this case, the person yelling was saying something entirely different than his words conveyed; he was looking for a fight and he might as well have just yelled that he was looking for a fight.

Good for you for doing what was right in your eyes!