Your story is somewhat similar to mine. I have often wished that something bad would happen to my ex. He was abusive, and so far it seems that very little "karma" or whatever people call it, has come back to him. I've been told that "he'll get what he deserves".
I don't want to have these feelings of wishing ill on him, but it just doesn't seem fair after what he put my son and I through.
I think the best reason to stop wishing harm on him though, is for yourself to forgive.
If we forgive what they've done, we can move on and not think about it. It is them that loses the power over us. He doesn't deserve all that space that he takes up in my head.
Your ex doesn't deserve to be in your thoughts, taking up that space and you reliving the hurt that he put you through.
Maybe things look like they are going okay or good for him, but you don't know what goes on behind closed doors.
Forgiving is for yourself, not for him. That is hard to understand, but if you really think about it, it takes away their power of continuing to make you feel hurt over the past.