I have kinda the same issue in a way, when I walk into my job you have to walk down this long overpass to get to doors to get in. I hate it. I feel very uncomfortable and cant stand the elevator, I will park on the fifth floor so I dont have to get in the elevator with other people. On the other hand it doesn't bother me quite as much to leave down the overpass. I wonder myself about this. I don't think you are selfcentered at all. It might be fear. Fear of not knowing if or when someone is going to be rude, and alot of people don't like crowds. Crowds seen very cold to me.