I don't feel any stigma from being diagnosed with Aspergers. Most people don't even know what it is.

I have to agree that people with Aspergers may tend to be easier targets for bullies but rarely become violent because of it. It takes another illness to drive someone to that kind of violence.