Who says we can't have memories before age 6? I have memories from the age of 2.

No, you were not the abuser...you were just acting out what had been done to you. Children tend to repeat what they've been taught.