I guess I might have said something along the lines of, "I use to know your former boss". That would have kept it formal enough to suit me and, presumably, her brother would not have been unduly curious or unprofessional to "dig" to a place where he would feel differently about you than he already does. I know I am "compatible" with what my parents, siblings, good friends, etc. do and some of them are therapists/assistants, etc. but it does not interest me that greatly? I have my own life and they have theirs. Getting further removed would be even less interesting?