Nobody should have to work in a toxic environment, but some people are luckier than others when it comes to what options (if any) they have.

Other people being unlucky is not your fault and is no reason for you to punish yourself unnecessarily.

If you are able to escape a toxic environment, and you feel ready to do so, then you have every right in the world to go for it.