Actually, a lot of people choose to distance from potentially toxic conversations or problems they see others experiencing. I have experienced that many times when I was treated badly.

One time I was being treated badly by a customer who did not want to pay me and was giving me a hard time. The neighbors who were friendly and nice to me quietly walked away. After the jerk walked away the people told me that yes the guy was a jerk and they prefer to avoid him.

I have had many experiences where my older sister acted out and even raged and other people literally distanced as they already noticed how badly she can behave.

During my life I have noticed a lot of people distance in different ways to avoid getting involved with conflict. I have even noticed that behavior in other primates. So it’s reasonable to think this is part of normal instinct.