There is fear of the stranger, the unknown, and I think prejudices rose out of that, as a shorthand way to deal with new situations. With less real knowledge and resources, primitive people had to respond quickly to perceived threats and if it is not "us" then it may be a threat. Shoot first and ask questions later. Attitudes for dealing with certain "others" were then handed down as Echoes details, through learning. Someone in charge's individual perception got accepted as fact about members in another group and that group and its individuals became outcasts.

I think prejudice is based on fear; fear that never got checked out well.