Maybe it's all just a flight or fight response. People are presented with someone that they can't understand/are afraid of/that evokes some kind of feeling of unknown/fear and they either fight you or run away. Then when they see you are willing to give a little they have to test how far they can go with you. They are doing trial and error on you to try and fathom you out.