How wealthy could we be, if we have dimes for every 'I'm only joking' we've ever heard?
No, I don't feel you are wrong for thinking that, about your grown children! Not wrong, at all. It's a living testament to the effects of mental/emotional abuse in the home. I think there's a thread down in Survivors of Abuse, that talks about why is emotional abuse dismissed or overlooked(or some similar terminology). Looking down in that forum, may help you out, not sure.
Just know, you aren't alone. Depression, panic, physical health symptoms, are all part of the effects of abuse.