Sex can make a relationship really confusing especially when you are young. I sure that he is probably just as confused as you are and it sounds like he was in panic mode. He probably finds your rejection of his apology just as strange.
The first time you have sex is rarely all that you anticipate and romanticize it to be. I suppose that it is hard but it sort of sounds like neither of you are ready for it. Sex with the man you love can be beautiful and a time of intimate communication that goes far beyond mere words. It really is best in a committed relationship though,Out side of that it can open you up to a lot of pain and hurt.
It will be better and life will be better, but as Harley pointed out above, this too shall pass. Remember that you are a wonderful precious human being and you deserve the best.
Hugs -- Larissa