It wasn't just bullying, it was so much more. I had so many female friends leaving me without explanation, and even my sister always preferred other children and later other people than me. I don't know if I am a good person, I guess I am average, not evil, not a saint. And, here when I think about it, the low self esteem started with maybe my sister and female friends, girlfriends, not really with men. Maybe they don't even see me as "sex thing" as I thought, I mean, they see most women like that, so why not me also? But the problem could be as you wrote insecurities they don't feel right away but pick up soon enough to be turned off by me.
I don't know at this age if I can really change that. I don't really have bad opinion on myself, but it could be that I am so desperate for a friend, that it reeks to high heaven. And since female friends all abandoned me, I seek that in relationships with men, and of course, that can not end well.
It feels to me I am getting to revelations here. At least something.