If they aren't kind to you and you can leave, do it.
This video I think speaks to that issue in the best way I have ever seen presented:
I have kids and can't leave my situation without causing them harm.
I have pretty much given up on the chance to enjoy my life, so it doesn't matter for me anyway.