I'm a man with Bipolar, and I have extreme hyper-sexuality amongst other issues. Unfortunately, prior to my diagnosis and subsequent treatment, this aspect of the illness made me a rather unpleasant person, a dreadful husband and I behaved in inexcusable ways. I cannot excuse the Behaviour with Bipolar, but it does explain it. When I'm euthymic (stable) I probably have a lower than normal sex drive, and I wouldn't entertain the idea of so much as looking at another woman, I'm more interested in cars and boats!
My honest advice would be to be very wary about entering a relationship with someone who has what appears to be unmanaged hyper-sexuality. Relationships and hyper-sexuality are, ironically, not good bed mates.
A side aspect of the illness is something I've heard referred to as 'Bipolar charm', and if hypomanic, one can become very charming and seductive which is dangerous. To my great shame, I have had several women fall in love with me when I've been in a hypomanic and hyper-sexual state. I had no interest in a relationship, and my behaviour probably resulted in them feeling how you do now.
Talk to him about his illness and try to understand how it manifests itself. Do you know if he has been diagnosed as having the illness, or if it's just his suspicion? As Wild Coyote and Ladyshadow say, it may not all be Bipolar, it may sadly be that he's a bit of a jerk when it comes to women's feelings.
I wish you every luck with it, it sounds like you deserve some happiness. And for what it's worth, now I'm diagnosed and medicated, the hyper-sexual, hypomanic states have gone and I'm a much nicer person for it, so there is hope, but one has to want to address the issues, and it sounds like this chap might not be at that stage.