I think part of it is also that the struggle is invisible. If the individual is visibly ill or disabled it is easier for people to understand they can see both the struggle and the cause for it.

With brain disorders it just looks like we are sad all the time or acting strange for no reason. There is no visual reminder that there is something actually wrong. It is not just depression sufferers that deal with this. I live in a military community and we have people who suffer from TBI or PTSD that have a hard time getting a diagnosis because they look fine.