I sounds like you've got a lot of pressure on you to "perform" which is making things worse. Depression can also cause it as well.

My suggestion would be to quit worrying about just having sex. Start to develop a relationship. Go on a date, be a gentleman, have dinner, be a gentleman, go to the club ... you know, meet a decent girl and actually get to know each other. Take your time and really become comfortable with each other and excited by each other. Then, when the time is right, there should be less performance pressure and things may go more smoothly.

You're still young, but you might also get your testosterone levels checked. Also, college can cause a lot of stress (at least it did for me), that can also cause problems.

So like I said, relax and start dating instead of just hooking up. When it's all said and done you may have developed a strong, loving relationship while your friends don't have anything but an itchy crotch.