I suggest you do tell your therapist what you have told me. What would you like to happen, though, if you did receive the diagnosis of asperger's? Would you like for your therapist to help you some more with these skills?

Keep in mind that there is a spectrum of autistic diagnoses. In fact, the term "asperger's" is going to be removed from the diagnosis list at some point. You could have a milder case, which might explain being able to have some friends (maybe when not as many social skills were required) but now that you are older, you are finding life more difficult. I can not diagnose, though. I am just guessing.