Could it be that your low self esteem is sending a message to females, who see something special in you, that you are not interested? Could you be accidentally sabotaging yourself with a belief that you will be rejected if you show an interest? Like a self protection mechanism?