Even if he has no intention of getting back with his ex, or is at all interested in her, it's pretty mean to the ex to spend so much time with her. I think the way he treats his ex's says a lot about how he'll treat others in his life... In my mind, he has complete disregard for ANYONE's feelings but his own.

I hope you are able to talk to him, and maybe come to some sort of solution, but remember to keep yourself as your priority. You deserve better.