I had people do that to me at jobs also. I honestly think that people can "sense" if someone has a mental disability, even if it's not outright obvious, and they tend to bully them.