Actually, thinking everyone else is looking at you and judging you is narcissistic, not a lack of self esteem. Other people are worried about themselves, not us. The way self esteem enters the picture is if one imagines the others thinking negative things. That's like a dream, coming from one's own head not anything "real".

I laugh at myself when I think others are thinking, looking, or talking about me. . . and then "change the subject" in my head, if only to start thinking about why I think/feel that and how such imaginings impact me. Am I looking for excitement? (think of something exciting to do) Am I looking for acclaim, for people to really look at/think about me? (think of something to do for someone else) Am I looking for connection with others? (think of a volunteer, hobby or other activity, something I have in common with someone else that I can work on and discuss with them), etc.