> I asked him why and he said it was because of how I looked.

I wonder what he meant...

Did he mean you are brunette and he typically goes for blondes?
Did he mean you are tall / short and he typically goes for short / tall?
Did me mean that you look depressed and lacking in self confidence?

Did he say anymore? Did you ask him what he meant?

I'm just trying to think up whether there was some sort of miscommunication thing like that...

Or quite what is going on.