No means no regardless!!
Doesn't matter if he thought you wanted to have sex or you actually wanted to have sex and then changed your mind. Stop means, Stop!! You shouldnt be questioning yourself on this.
He is very much in the wrong. I don't know too much of what happened so can't say if it's rape but I would say this is sexual assault. Are you ok?