Marigold...you are NOT being possessive. What he is doing is just plain rude.
I have to add, IF you feel you are really in love with the man, you may have a very hard time, either if you detach yourself from him, or you keep going with him. Either way, it will take a lot of emotional effort on your part. Staying with him, ask yourself if this is, indeed, the type of man you want as a partner.
Honestly, you don't owe him anything...nada. You can walk away from this (no matter how intimate you've been) and go on to meet other prospective partners. In my naïve beliefs in devotion and commitment, I struggled and hung in there WAY too long. You don't have to do that, and you don't have to engage in a lot of long explanations and apologies. Just be done with him and move on.