The first few years after I was diagnosed I avoided relationships (of all kinds) due to the reasons you mentioned above, all of which added up to low self-esteem and self-worth.
Loneliness and isolation took their toll and it became a vicious cycle. Gradually I realized that BP is a condition, not who I am. I decided I had a right to companionship and happiness.

I have poor social skills, but I carefully began to make friends and accepted the few dates that came my way. I did not disclose to anybody, male or female. My feeling was that the stigma of mental illness is so pervasive that I would be ostracized and scrutinized. I became (and still am to some extent) a "closet bipolar".

I developed a relationship slowly with a man who later became my husband. I did not disclose my illness to him until we began discussing marriage, fully expecting the relationship to end. We have now been married 24 years. In retrospect, I would have not had children, since now 2 of my 3 kids are showing symptoms. The prospect of them suffering the way I do is too much to bear. but that is another discussion...

I am not an advocate of blanket disclosure. It's only fair to myself to allow relationships to grow slowly, so that people can get to know me with all my faults without viewing me through the lens of MI.