Yeah I second keeping a few notes of what was said by whom and when, hopefully you won’t need to raise it but if he persists and you do it will be very helpful to have your facts in order.

Try to keep cool and like Divine suggests say you prefer to focus on work tasks only if he tries this again.