3 to 4 months really is a long time and he probably is going to continue to press. I don't want to doubt you here because I'm not there but you really want to make sure that you are loud about this. It needs to be clear that you are angry and you won't allow it anymore. You need to verbalize that you aren't going to let him do it again. Not only is this to make your intentions known but to make sure that there can be no denying that you tried verbal defence first. When you push his arm away do it with force and not a simple swatting motion.

If he tries a second time then defend yourself because that is certainly assault when he does that. If he has friends be careful because guys these days are wimps and sometimes jump in so be aware of his boys if they are around. And keep off the ground whatever you do.