I think it's because for a high-functioning adult (my perspective), getting a diagnosis won't get them any help (or very minimal help), so people don't see the point in getting a diagnosis. What they then don't understand is that having a diagnosis allows one to understand and accept themselves.

Another prominent reason is that people think the label will prohibit you from getting a job. I think this is just a myth because no one requires you to disclose your diagnosis for the vast majority of jobs.