Sometimes, even if people are aware we have psychiatric issues, they don't classify US with "the nutters." They don't think twice about throwing words like "crazy" around in our presence because they don't consider us "crazy." Why? Because even with a diagnosed illness, we don't fit the stereotype of a "crazy person." The OP, for example, obviously has a job, can talk sense, etc., while people tend to believe a truly "crazy" person cannot.

I see the same thing as a plus-sized woman. Someone may make a remark against "those disgusting fatties," and if I say something about my own size, they'll tell me I may be big, but I'm not what they consider fat. What this means is, "You aren't lazy, stupid, smelly, ugly, sloppy, constantly stuffing your face, or other stereotypes I associate with being fat."

Another parallel, my husband considers homeless people "bums" and refers to shelters or slummy neighborhoods as "bumville," even knowing that when we met, I myself was living in subsidized housing, and before I got that apartment, I was homeless. He wouldn't think of me as a "bum," though, because to him a "bum" smells of alcohol and drugs, dresses in rags, would reject a job if offered one, would repay your generosity by stealing from you, and other stereotypes of homelessness.