I recently remembered in more detail something at a family reunion when I was barely 3. Other things happened after that, but my T always wondered if there was something earlier because of how I reacted to the later incidents.

We started talking about my fear of clowns, and my irrational but deeply believed belief that they eat people. Then.... BAM.... I started remembering specifics from the event. I am 44.

SO yeah something you don't remember can effect your behavior and emotions profoundly, then suddenly be remembered. It isn't like remembering makes it worse necessarily, but sometimes it helps explain things.