Just my opinion, but I too think you were being far too kind. That said, you seem to behave the way that is best for you, and that is perfectly OK. You need to do what is best for you and I think you are finally prioritizing yourself and what is important to you.

Others will find a way to accept your choices eventually. If they don't, perhaps they are getting what they deserve for the damage they caused.