I can't speak for the US but in Canada our curriculum in both elementary and high school includes world geography as well as world events. I remember my son studying maps of the world in preparation for tests a couple of different times the last few years of his education. He has also prepared speeches and written papers on historic and current international events that have stimulated for him an interest in travel and different cultures.

I think low scores say more about how much people pay attention to what is going on in the world and or how much they have travelled around the world. If I depended solely on my memory from school days I think I would have quit before I got to the second level.