From someone who is naturally very selective about who she decides to be open with, or with whom she decides to foster strong connection:

It can take me a very long time, often, for me to be as comfortable with other people as they are with me. I am usually pretty cautious and tend to be pretty gradual in how I let people in. To some people this is obvious and to some not.

Because there are certain things that I am pretty free and open with, many people don't consider that I am often afraid to be open with people in a general sense... frequently I get a similar frustrated response as the OP has described.

I don't know if this is the case in your situation, Dylan, but it is one possibility that you might want to consider.