Oh, my, Blue...you have describe quite a history here. I'm speechless, to be honest.
I'm going to suggest finding a quiet place within yourself...step back from it all, if possible, and assess what you, I mean YOU want in life. You have had a difficult time with men. This can leave one bitter, untrusting, and deeply wounded. The children you mention have to have suffered from this also.
Remember...quiet yourself internally, even if it takes several days...Don't think about what HE'S doing. Focus on yourself, and find that quiet place. Then, once you're there, start to asses the situation more objectively.
Love
Patty