It sounds a lot like you're nervous, which is an easy way to lose "the mood." The more persistent your anxiety about it is (meaning you're nervous even out of sexual situations), the more reduced your sex drive is going to be, I would suspect.

Just relax. Let it happen as it happens. Nothing at all is "wrong" with you. You're just nervous, is all. Losing your virginity is often viewed as a major milestone. Anxiety is natural.

Once you become more comfortable with this girl, I think your problem should cease.