Not silly at all. My abusers trained, taught and programmed me to feel guilty any time I did something like not have contact with them, talk to someone about them, tell what they did, etc. Keeping me feeling guilty and questioning what was wrong with me, kept them safe from me telling anyone, and kept me connected tot hem so they could continue to abuse me.
Feeling guilty is a nayural thing for survivors because they have taught us to feel that way to help keeo thier secrets. The fact that you have walked away shows young strength and your healing. Way to go! It takes a while but eventually we grow to the place where we can put the guilt and blame on the abusers where it belongs.
You are doing great!