It's called infantile amnesia - the brain is not developed enough to store long term memory of events that young. It is indeed not considered possible. And you can never exclude the possibility that the one remembering is actually remembering what someone else told them about their birth or infancy - which would be why third-party accounts agree with the memory.

I think there is a school of thought that there can be somatic memories from a very young age, as another poster said. If a baby is left alone at night to cry, that can become a very strong somatic memory. And maybe a difficult labor could become a somatic memory as well. Is that what you're experiencing, OP?