Oh man how does one answer without causing extra worry?

Yes it actually is possible for young people to have dementia. In several cases when it happens it is a vascular issue, for some reason or another, blood will not go everywhere it is needed. There is also a chance of vitamin dependent dementia even in the younger (vitamin b1). And there are rare cases of very early onset Alzheimer's (worked with one when I was young, she was 28 getting sick). Also there are rare forms of dementia that basically are unheard of even in the elderly, rare does not mean totally non existing though (those cases are usually genetic).

As for genetic early onset Alzheimer's there are genes found that are thought to cause it, which can be identified. One type of inherited vascular dementia can also be identified by genes.

Dementia can hit most "any" part of the brain first, which means it can cause a wide range of neurological symptoms. It can be memory issues, problems with finding your way, problem with carrying out tasks, problems with focusing, problems with language and so on. I'd think if a doctor does not see a decline, they would not think it is dementia. Also just a standard GP could run a quick neurological test, which might catch something.

There are a lot of things that can happen with brains, even outside dementia and mental illness, such as inflammation brought on by an immune system response, something secondary to an infection etc.

I hope the doctors can ease your worry. I know though that it really doesn't make one feel better just hearing something is not very likely. So hope the docs can give you a more definite answer.