I think for his birthday you should go out with your women friends and have a fun alcohol free night and forget about this man. And it's just not a good idea to flirt on the job. Anyone who is truly interested in someone can figure out a way to make contact outside of the job. This man sounds like he liked the attention but didn't want it to go further. Find yourself a trustworthy available man.