It sounds plausible based on your post. Not that it means you don't have it, but Autism/Aspergers does affect males in a 4:1 ratio over females so that explains why so many of your male relatives might be on the spectrum.

There's a lot of stuff to read online that you can use to try to judge better if you think it applies to you. If you decide to look into a diagnosis you should definitely try to find a psychiatrist in your area who specializes in the autism spectrum.

Thinking back about how you were as a child can be useful too I think. I was a very odd child, had some weird rituals, repeated odd phrases that I thought were amusing, collected things (carved wooden and stone animals for example - imagine a 9 year old spending his $30 savings on a wooden carving of a rhino ), bad socially, sensitive and withdrawn, stimmed (still do but not as much), felt disconnected from people, never really "got it" in a general sort of way...

Anyways, enough about me. Hope that was helpful. I am not "officially diagnosed" either. Good luck!