I think its a bit rude of you to maintain the relationship with the woman you cheated on your wife with! Regardless of the past, regardless of what your wife SAYS, it shows a basic lack of respect for your wife and your current relationship. You are openly welcoming temptation into your life. Why would you put your marriage at risk like that?

Its hard for most adults to make friends. Everyone is busy with their partners, kids, jobs, etc. I mean its not like when we were back in school and everyone had free time to pal around and it was easy to make the kid who sat beside you in homeroom your new best friend. Please don't make that your excuse for not going out there and finding someone else to be friends with.

Always had a mutual attraction coupled with cheating on your wife with her.....this spells trouble for your marriage. (I somehow doubt you've told your wife about the mutual attraction bit, am I right?)