I have always struggled with this feeling as well. I have a bad habit of knocking myself down further, by telling myself that no one is bothering to look at me, so I should get over myself already! (Not good for my self-esteem, I know) But, yeah.
I feel very uncomfortably self-conscious. Unless I drink a few beers, that's when I loosen up. Not a good fix either, huh?