I think the complete opposite is true.

As the others have said, when people are feeling their avoidance most, they do everything they can to blend in and be pleasing. They will go to the ends of the earth to make sure they will pass muster, and fuss over every aspect of their appearance.

Plus - many people? Normies you mean?

I've been avoidant for decades. I have never, never once had someone call me out for being avoidant.

First of all, "most people" don't even know that AvPD exists. And secondly, I do everything I can do to pass. I stay as quiet and in the background as I can, but if I'm confronted, I can fake it. I fall apart afterward, but I can talk to people if I have to. I can't imagine that some random person who didn't know me really well would ever think I was anything but a little reticent. It kills me, but I make sure that I blend in.

Again, it's common for DEPRESSED people to not tend to themselves. And, yes, there can be a general impression that depressed people don't care about their appearances.

You're mixing things together that, like oil and water, can't be mixed. Don't glom everything together. Just because you experienced something at one part of your lifetime does not mean that it's automatically an avoidant trait, and that every avoidant shared that experience. On the contrary, it would be unlikely for you to ever find someone who had an exact match of your background.