I definitely understand what you mean, I got my diagnosis when i was 14 and i've been dealing with it for 6 years now. In that time I've told my best friend (whom I consider a brother), my roommate, and one girlfriend who came with me in the hospital the last time I was there.
I'm planning on telling my "parents" (I live with my best friend/aforementioned brother and his parents) and tonight I'm actually going to tell every person in my fraternity... saying it in a room with about sixty other people will definitely be the hardest thing i've ever done. Even when I talk to people that know I still don't ever actually say what "it" is.
I've found that every time you say it, the next time gets a little easier. It's a matter of being comfortable with yourself and being comfortable answering questions about it when people have them.
Best of luck to everyone!