Have you tried wearing sunglasses with these people? I have transition lenses that take a long time to change, and i really think i get less shyte from people when the lenses are dark.
Try getting some clip ons or the big overglasses they give you at the eye drs office? And see if that helps, if these people who bother you cant see your eyes, but you can see theirs.
Or a hat with a brim, even a baseball cap.