The way this guy lives may be what's right for him. That's what I learned at Al-Anon: how another person chooses to live is their business - even if what they choose is dysfunctional and maladaptive. That's on them. If we believe in free choice, then people are free to make poor choices. You can't choose for someone else . . . only for you.

I think you chose well to bow out of this relationship. It would do nothing but keep you perpetually irritated.