Sex should not be important at this point. Setting boundaries and working on communication is a must. If he is going to have similar reactions like the breaking up again, its more important to create that boundary that prepares you for the event. He needs to know what is and isnt appropriate. Then you should work out plans on how to approach those problems. Counseling would probably help.

Honestly, dont allow him to make you feel unattractive and dont feel afraid of saying no. You werent being too sensitive and I would feel hurt as well.