Ah, that explains why being in unfamiliar surroundings causes such distress. Perhaps there are precautions which you could take to put your mind at ease? Pepper-spray, for example? It is also good to walk nearer to well-lit, populated areas. Does knowing that you are doing things to keep yourself safe help in any way?