He recorded you? Great.

It doesn’t matter what happened in his marriage. He’ll likely tell people that you were abusive too. He now has recordings of it. I’d not give any thought on his ex marriage. Not relevant

I think you both talking to each other right now opening up possibility of a very toxic cycle. Could you try to stop talking to him or it’s too hard?