I say be careful. I'm not saying definitely don't do it. Use your judgment. It could get complicated and if it ends and gets awkward to see him at work, are you willing to quit your job and look elsewhere for work?