That's a bit like saying that being short isn't to do with your height but your body's ability to grow tall.

It is an attention disorder because it is a problem with your brain deciding what to attend to. I'm not certain if there is a way of quantifying attention outside of this context. Can you have more or less attention as a discrete value set apart from your ability to determine where to direct it?