You mentioned that you haven't touched on previous harrassment/gropings so maybe he was coming at it from the angle of how you present yourself as either professional or flirtatious?

I don't think it necessarily HAS to be blaming. Maybe he doesn't understand the context?

Like say a woman goes to a bar. If she is overly flirtatious, chances are that she is going to get hit on, right?

I'm glad you're going to talk to him first. So many things can be misunderstood.