Yes. Asperger's is required to start in childhood; it usually starts in infancy, and the rest of the time by the toddler years, though it often only becomes apparent once school makes social and executive-function demands that the AS brain has trouble handling. In fact, if it starts in adulthood, that's a reason to suspect something else, such as social anxiety disorder or schizoid personality disorder.

However, it is not unlikely that your mother believes that you didn't have AS as a child not because you actually didn't show traits, but because she has a mental idea of "autistic" that you didn't match as a child. The stereotype of autism is a picture that very, very few autistics actually match, though--and that means that if your mom is comparing you to that stereotype, she may be saying you weren't autistic as a child, when you actually were.