Frankly, it is hard to know how a person will turn out, so please don't be so hard on yourself.
Have you ever asked him if he would go into marriage counseling with you? It sounds like he has some serious issues. I honestly think you and your son would be better off without him unless some major changes occur, and I don't see much hope for that.
I am glad you have your daughter to live with. I assume you will be seeking a job yourself and will be helping your daughter in some way.
I am sad this relationship turned out so badly.