I can relate to feeling lost without many memories of growing up. Most of my childhood is blank except for some negative memories, and more recently, some pretty intense flashbacks. I still don't remember much of it when not in the middle of a flashback. It took a lot of years to get any of it back.
If you are working with a therapist, perhaps you can focus on how what you remember (or don't remember) effects you in the present. I wouldn't suggest digging for memories, as what comes up can often be unreliable as well as overwhelming.
It's really frustrating having all those holes though. I hate chatting with people and having no recollection of what they are talking about, or not being able to give an anecdotal story of when I was younger. I've started just nodding and agreeing with stories people tell of things we have done together. I had recently gotten back in touch with a really good friend from my middle & high school years. She would recount storie of things we did, or ask if I remembered doing soemthing specific. At first I told her I didn;t remember, but maybe if she told the whole story it might jog my memory. She started gettign frustrated about it, taking it personally (like none of it meant anything to me b/c I didn't remember). I ended up having to explain that I don't remmeber most of my childhood outside of some really crappy memories that I would honestly rather not have. Now she tells me the stories of things we did together as if she were telling someone who wasn't there...
Hope you can come to some terms with what you remember.