Ime, sometimes we find a person irresistible when they remind us in some way of a person from our past. In my case, unfortunately, it was the way they reminded me of my mother (happened with both males and females) and were ultimately unavailable for a real relationship, but still would try to get something from me.