Good for you for not running to someone else this time. Before when i would have manic episodes i would have sex with anyone. It to time for me to learn not to turn to sex to relieve my manic mood. I had to really learn to avoid situtations that gave me access. Such stop communicating with ex-lovers. You have already started this process by avoiding texting while you are depressed.

Now to address the withdrawl during depression. I do that too. But when I'm depressed my husband understands I cannot be as present as I can when I'm normal. You on the other hand have rebuild the trust that has been broken. When you are depressed you are going to have to force yourself to give her some sort of affection. I know that this will be hard be will be worth it for your relationship.
Hope this helps. Hang in there and stay away for texting. Good luck.
Gayle