I don't know what to tell you. I agree with the above, except for trying to go back to your ex wife. I imagine she is devastated over the affair and over you leaving her for another woman. It does sound like you were in the moment with regards to this other woman and didn't really know her that well before moving in together.
You could have chosen to work on the marriage instead of having an affair - so why didn't you do that? You don't have to answer me.... I am just curious why you gave up on your marriage without trying counseling first.
You also must have known prior to marriage that your wife didn't share the same passions - so why did you get married? This is also something that could have been worked out, possibly, had there been conversation around it. Your wife could have joined you in sharing your passion, even though it wasn't her own. That's what couples do - they share.
But now I'm afraid it's probably too late. I would not take someone back if they left me for someone else. That's like a spouse thinking the grass is greener in another pasture, leaving and then discovering it isn't.
I would give up the current relationship and get professional therapy to figure out why you did what you did and what you can do now.