This is something you must decide, and also weigh the risks, for yourself and the unborn. Is there anyway you can be just satisfied with your existing children? I was able to only have one, and I adore him, I went through a period I missed not being able to have another one, but I got through it, and life continues, I do not want to sound cold or minimize your concern, but coming from a person now 47 with only one child now 17, I am thankful for at least one child I was blessed with. This is truly a choice of your own, you will hear different opinions from others, and some you may not like, so be prepared that this may happen.
Yes, Bipolar is genetic, and can come in many different severities to mild, there is also treatment, not a cure but putting things in a "remissive" state.
The next thing is to ask yourself, do want to risk going off meds. and having things go awry? Do you want your existing kids have to see you in this condition if things go wrong, being off meds? Another thing, when meds are cut off know one can be sure if a potential episode(s) will occur, and some can be more severe than other times before, it is a extremely hard decision, I personally (being Bipolar myself) would for go it, try to get involved with something that can be rewarding, true it isn't going to replace having a child, but in time it helps.
Again this is your own choice, so many things to think about and the possiblities of things that can happen, you may even want to discuss this with your doctor(s) and a therapist, perhaps they can give you some input about this.
I myself would be satisfied with the fact I have the child or children I have already, and not take any uneccesary risks, that would negatively affect myself or family.
That is entirly my own opinion, so do not let that insult you or offend you, this is just my own personal feelings.
If you are in deep question of what kind of outcomes there could be you and your husband need to really think very deeply about this, even go for counselling before getting pregnant.
I wish you lots of love and luck with this, it is such a difficult decision I am sure, so it is important not to run into anything and then have regrets down the road.
Please take care now,
DE

In giving advice seek to help, not please your friend
SOLON