I think it depends from person to person. If you both go into it as we're going to have sex but this doesn't indicate we're a couple and we'll still be friends - then it would be okay like Tishie said.
The only problem I see happening is - this is a guy from work and guys talk and I wouldn't want you to be where the other guys think "well maybe she'll have casual sex with me too. That's the main problem. Are you sure he doesn't have a girlfriend? - I wouldn't want this to hurt another woman.