It would be a red flag for me if my boyfriend chucked labels at me, especially as you said he was using it to invalidate what you were saying:

"I'm not taking the word of a person who has schizophrenia"

Seriously? Not okay on a number of levels.