Have you considered asking him if he'd like to get coffee or a drink after work one day? Maybe he is having just as much trouble reading your signs and isn't sure whether your giving him your number was a sign that he is encouraged to call you? I know that sometimes, I am so receptive to other people's emotions and actions but I don't realize that I am closed off and mysterious. I am really really bad at expressing myself, especially in that awkward "does he like me?" phase and I come off super stand offish.

Not saying this is what is happening here, but I think you need to be more assertive if you want answers.

As far as office/work romances, I don't think that you should immediately dismiss one just because it can be challenging. Honestly, I meet most of my friends through work because otherwise I am very antisocial. Is it difficult? Yes. I have worked with two partners and neither went entirely well, but we were very clear on boundaries. It is more difficult if you work DIRECTLY with one another versus in the same building or for the same company. I know many coworkers of mine that are dating or married to coworkers. Just obviously steer clear if either of you are in a position of power/authority over the other in your work roles.

I think you might just casually mention a neat coffee shop or restaurant one day when you're talking with him and then ask if he'd care to join you. You can make it as casual or formal a request as you want. Be clear with your intentions if you want him to be clear with his. If you gave him your number without any real indication of what that means, he might be confused. If I were you, I'd rather know for sure before I dismissed the entire thing entirely. I've had dudes blow me off and I had to do a bit of investigation to make SURE they were blowing me off and it wasn't just a miscommunication.

You talk a lot in the past tense do. Do you not see him at work anymore? How long ago did you give him your number?