I think peer pressure is greater as an adult than it was as an adolescent. When you reach maturity people expect you to be connected to them and everyone around you. If you're not open or connected to those around you people consider you self-centered, selfish, ignorant, etc. Being connected to others means you absorb what they're feeling - when those people are distressed you're going to feel it. I guess that's what a free spirit means - free spirits strive to connect to themselves, have solid boundaries around themselves and don't take on the stress of others.

I'm sure there are self-help books out there that could give you advice on lowering the stress. Psychiatrists, therapists know how to keep boundaries and not take on the stress from their patients.