Considering your feelings and vulnerability right now, can you insist that he go back to counseling with your or on his own? Let him know that the future of your relationship depends on a change in his behavior. If he refuses, you can begin to seriously contemplate leaving after the baby is born (give or take a few months). You don't sound ready to walk out now and, if you don't have any support system aside from him, start working on reconnecting with old/new friends now. This is important for you and your daughter. You have to think long term planning!