I feel like I need to be blunt. You probably know this info on some level anyway. I don't think the two of you can ever be "friends." Perhaps he is hanging on to you in case his marriage doesn't work out. But it's not appropriate in my book for someone to be the "other woman." Plus a man who would cheat on one woman is likely to cheat on another. You now have an enemy in his wife, and I wouldn't think you would want to encourage your reputation. He is breaking his vows to her.

I wouldn't respond to any attempts to him getting in touch with you nor try to see or contact him again. If you need to, then see a therapist to help you break away. Then go out with girlfriends, have some fun, etc. Maybe even try to forget men for a bit. After awhile, then you can look for a man who is decent and will love you and you only, and not make you feel like a boomerang.