Remember, as cliché as this sounds, it isn't about you!! You are affected, yet, comparing yourself(in your mind) to the other woman, isn't the reality. The 'other woman', in my life, is perhaps 5 or so years younger than me, but looks older than me. umm, that in words, written may sound bitter, it's not. That would make her, about 17 years younger than him. She's a married woman.
My ex has serious control issues, etc, stemming from childhood, etc. He is an alcoholic, as told by the primary care doctor, he and I went to, while married.
He was abusive to me, in marriage, and even in divorce.
Point being, there was some void within him, that even a loving, caring, compassionate, do everything for him type of wife couldn't fill.

My grieving the end of my marriage, occurred during the marriage, not after. After, was almost a sigh of relief. Almost because there are children involved.

It's too easy to sit and feel badly about ourselves. Wondering, what it is about ourselves, that can't be good enough for the men that we thought loved us forever. He said many of the same things.

Even in divorce, from him, I get comments such as I've destroyed his life. Perhaps, what was destroyed, was he could not longer carry on as a married man, in society. Hmmm, before my divorce, I had to ask myself so many of the same questions that you are asking yourself.

I reached a point, where I said, I have had enough. I will no longer allow this man to have so much power over how I see myself. My self esteem was worth way more than trying to please a man, who just cannot seem to get it together.

Sure, there are many lonely nights. Sure, single life isn't always all it's cracked up to be, but letting go of that hold he held over me, that was worth it.

Now, that he's not part of your daily life, what outside interests are you pursuing? Hobbies? Any girlfriends to go to the movies with? What can you do now, that you don't have to answer to him?