It is wrong to judge people purely on your subjective perception of their physical appearance, yes.

I'm sorry if anyone has made you feel ugly.