It makes you vulnerable. It's not just two people looking at each other, it's a deep communication. You can look into a persons eyes and "see" the person. Words are just intellectualizations of thoughts and feelings, and thoughts and feelings can both be read via eye contact. If you read this article on the power of eye contact you'll see what I mean, it's powerful stuff.
The Power of Eye Contact ? Make Someone Fall in Love With You! - Psychologia