There is a man at my work who is romantically interested in me. He's not said it to me.... Yet, however colleagues have told me he's expressed and interest in my 'dating life' and was asking if im seeing someone as he wanted to take me out.

I've got no interest in him whatsoever. To me he's just Jim who I work with. That's it

I'm worried I'll be caught off guard at lunch time one day and he'll ask me out.

Rubbish at this type of thing... Advice?