I have read this entire thread, and am glad to hear you won't settle for an affair.

I find it a little troubling you want to apologize and also tell him how you feel. That puts you in the position of "waiting in the wings" should his marriage end. What a lowly position to be in. Yuck!
While not directly interfering in his marriage, it is providing enticement or incentive as there is a replacement waiting.

While I hear you say you don't want an affair, I will share what did happen to two women I know who did have an affair, and the men left their wives and married them. The outcomes of both were remarkably similar.

Both men lost their relationships with their children. The children would not even speak to the new wife, who they blamed 100% for the breakup of their parents marriage. That may not be true, but it is how they perceived it. And the fathers now have no relationship with their grandchildren. They missed their children's graduations, marriages, pregnancies and other accomplishments and rights of passage into adulthood.

One woman was even referred to openly as "The Witch".
That is basing a new marriage on great sadness.

One woman found out years later that she was just one of many he had had affairs with. She eventually came to see herself as gullible for having fallen in love with him, and there was very little joy in the marriage due to all the pain caused to others that had come as a result of their falling in love. And she eventually blamed him for chasing her and causing her to fall in love with him.

Kind of like what you are saying. He chased you first.