Why do you say it isn't a disability? I think that anything which interferes with enjoying and living your life is to some extent a disability.
Depending on your line of work it could even affect your ability to earn a living, can you imagine a lawyer or real esate agent who had a problem being around people? Being downwardly mobile as I am, my people problems still hurt my income. While I'm now in food service instead of an office, I make very litle money as kitchen help wheras if I could stand the human contact I could make as much in a day as a waitress as I make in a week!