I found that it doesn't so much depend on the people, but on what I do. I had to learn some social behaviours to fit in, and what behaviours to expect in response and how to interpret them. Very hard and unnatural at first, but seemed to be the only way for me, and it does get easier over time. There will rarely be a person to break through my barriers, someone who just loves and is fascinated by all people.