I can understand that because you work with him you would want to be indirect, but there are a lot of ways to be direct with him but keep from walking that "sexual harrassment" line.

You can ask him if he'd like to go with you to the coffee pot and get some coffee...

Maybe he'd like to sit next to you in the break room...

Maybe he'd like to go with you to get some water from the water cooler...

Would he like to go with you to the soda machine?

As you can see there are small ways that you can draw in his company without crossing any lines.

If he's interested in you, by making these gestures, it tells him it's ok to move forward and maybe ask if you'd like to grab lunch at the local restraunt...

Anyway, you'd know!