I think it's less likely right now that inviting someone over to watch a movie means having sex. I don't know if movie theaters are open in your area, and I assume restaurants are, but even so, lots of people are still choosing to limit those kinds of activities. One of my parents used to go see at least one movie per week and eating out or having a coffee in a cafe were frequent activities. She hasn't done either since March.

And like lemon said, you are allowed to say no for any reason. Anyone worthwhile will be willing to wait and not hold it against you. I do think many guys will take sex if they think it's being offered or you might be amenable regardless of their long term intentions. I wouldn't necessarily rule out a guy who makes a move this early, so long as he isn't pushy and accepts your refusal gracefully.

ETA - And I agree with LT about not informing/accusing him of inviting you over with the intention to have sex. That just introduces unnecessary tension and puts him on the defensive. Not a good start to a date.