Some if what you are saying is really similar to social phobia. The worrying about what people will think, and avoiding situations because of it, and being embarrassed about your lack of contacts.
Can you deal with this in small ways, just as social phobics have to learn to do, by pushing yourself just a little bit, and going to one thing you are invited to? When you are there, focus on other people. With social phobia people think about what others might be thinking of you, so make yourself think about what they are wearing, ask what they have been doing, and really listen to them. Take your focus off you and whether or not you are fitting in. People really like to be listened to.