Well, considering the fact that this man is your supervisor or boss, and this other yougner coworker expressed an interest in you at first, there could be fear of workplace relationships. It could be that his bosses or co-bosses advised him against personal communication with a subordinate. Work ethics can be tricky and he might have overstepped the boundaries and been told to back off.