If we were punished as children, and other children were not, doesn't that mean that something about us was wrong and that we deserved punishment? I mean, how else can we explain it?

There are, of course, other explanations. Just not ones that easily come to the mind of a child...