I'm sorry you are going through this. Do you have a therapist or psychiatrist you talk to? If so, have you bright this up, or could you? Memories are a tricky thing. I think after a rash of false memory cases, people are hesitant to follow up on stuff like that, especially if it Is no longer occurring. Having someone that knows you, or is willing to explore the possibility with you might be good. Also, definitely exploring the social thing would be good to do with a professional. Even if you don't ever uncover a specific memory, talking about the things going on now is a place to start.
I know it can be super frustrating not to have concrete answers. Is there anyone you can confirm your suspicions with? Would anyone have known?
I dealt with similar worries a long time ago. It was confirmed to me that something had happened in the past, but not until the guy was released from jail (I was still a kid, and told to call my parents or the police immediately if he ever tried to contract me). Maybe there's a family member, friends, or someone from the past that can confirm or deny your suspicions?
Sometimes deeper connections with a population can come from other sources though.
Again, sorryyou are struggling