For me it's social anxiety, and the fear of accidentally looking at someone the wrong way or for too long and offending them somehow. It could be due to social norms and not wanting to cross invisible "boundaries" regarding strangers in public places. However it's so automatic and I don't feel like I can help it. It actually bothers me that I do this, but I've always been this way. You're lucky that you don't.