Does he compliment you on your looks? Maybe your doubts are justified. Every guy has a type of look that he prefers in a woman even though he may settle for another if he's lonely and not good at going for what he wants. Of course, the type of look he goes for may change in some regard if the right woman comes along. If he openly insults your looks and doesn't compliment you, then maybe you should let go of this one though.