From my biological parents unintentional neglect, cause they were both schizophrenic and unmedicated.
From my grandparents, they just didn't like me too much to care, maybe they saw that I was odd and said I was just like my parents. They never said crazy or mentally ill, but I assume that is what they meant.