Sorry - I just re-read your post. You also suffer from BP.

Well, I've been there; a relationship between 2 BP people and it really is not easy. Your cycles tend to run very different and therefore your needs are constantly changing. You may not be in a position to offer him what he needs right now, or he may be going through a manic phase, having little regard for anything or anyone.

Give him space. Maybe therapy for the two of you will be beneficial. Communication is crucial in any relationship, but even more so when BP is involved, so you need to constantly be aware of where in the cycle wither of you are. You need to lie down boundaries and rules as to how to handle each of your cycles and the various episodes.
It's going to take a lot of work, but a relationship is usually worth it.

Again I stress - don't take it personally; just offer as much support as possible, being careful not to make him feel claustrophobic.