I don't know... my husband is rathe good looking as well, and I'm rather not at all. Like... there is a mis-match with us appearance wise (but not personality wise.) We met and fell in love online with no pictures at all, until much later. So, I always say he fell in love with me as a person and not my appearance. But, he disagrees.
But, I get that same feeling. But he really is attractive and it is very likely girls like him. And that's fine. They can like him but they can't have him! As long as he is not reciprocating, then let them fume and wonder why he's with me, because "haha," (best Nelson voice from the Simpsons).