Please give me some insight or advice

Cutting it short my "self-made" diagnosys: avoidant personality disorder, schizoid personality disorder, social anxiety, depression, dissociative disorder... So here it is my description.

Now the story: One of these days I was invited to hang out with a colleague I used to attend university with. We were never friends (as I just don't have any friends), but it was one of the few people I could almost carry a conversation with. Not that we talked that much, with did some group works together, but we never hung out outside university.
So seeing a message from him inviting me to hang out, after almost a year without seeing each other got me very confused.
How could I say no? Or did I have a real reason to say no? A lot went through my head, specially about what his intentions are (I haven't figure it out yet), but my reasons to say no are all out of "fear" and how would I know they are real if I keep avoiding things.
I felt stupid if I sayed no, because I allways say no to everything... Well, I told him it was ok, I tried for him to call some people to join us he said maybe, but then said he wasn't in the mood to have the "work" to organize something. Know he asked me if I had someone I would like to bring. Of course not, have I mentioned I don't have friends? How I wish I would. Why is he asking?

There are a lot of mixed feelings inside me, so many doubts. I don't know him deeply, but I know his character and appart from some anoying things he is a nice person.
I live with my sister and I have scheduled a day she isn't in the city, to meet him, I don't want to justify myself to her, she would have so many questions and she would make me feel like I have to tell her I don't care about this person, she would have suspitions and would made me feel in the spot light. I don't tell her my deep feelings, I don't talk to her about my disease because I am embarassed... I am afraid this might be more than friendly. I know I must be over analazing something normal, but for me is such an abnormal situation. Am I being naive? Am I being paranoid?

I never went out with someone other my family in a one a one situation. I haven't went out without my sister (older sister) in years. I just go out with her, and with her and her friends, to whom I don't talk much.