I think if you would have opened up and told your story than it may have given you a hard time in the after math. They didn't need to know your business anyway. If you had "passed" as a child of privilege than you must have fit right in with those who were raised "normally". That speaks volumes about how you're able to conduct yourself in spite of not having a normal or privileged past and your not speaking out of anger is apart of that common sence mechanism that made that/those person/people assume that you were one of them. That's not a bad thing. I think the people who ARE at the place you came from need to know your story moreso than the people who've already had made it to the university anyway.
PS. KUDOS!!!! You did great!!!