I don’t think it’s necessary to tell them your diagnosis. That’s your personal business. I would only tell someone if I wanted to and in most cases I personally wouldn’t want to.
I think they’re making comments and asking questions because if they are anxious about something they can’t wrap their minds around someone not being anxious. I think it’s just curiosity and if you don’t want to make anything out of it, then just don’t. I tend to just tell people “I’m just weird” when I don’t want to elaborate on something I do that’s different from them because in many cases I really don’t care to talk about things with people.