Their marriage is not your responsibility, it is his.

Their marriage is not your problem, it is his.

The guilt here is not yours, it is his.

You did not betrays someones trust. He did.

He made his bed, and he will have to lie in it.

You followed your heart, not your head. Your way out is to do the opposite. Follow rationality. Follow logic. Follow your head and ignore your heart.

What I think you should do is find distance. Such a relationship is doomed to fail. You are setting yourself up for emotional chaos. Much greater than it already is.