Sex can be such a tricky thing to navigate to make sure that both people feel comfortable.

Are you able to talk to him about how you are feeling and what has happened? There may be a good chance that he doesn't realise that it upsets you.