Listen to your feelings. I work at a hospital too and had a similar experience with a co-worker...He never actually tried anything physically with me...just stared and lurked around wherever I was. Something about him made me go out of my way to avoid him. He did end up getting fired for trying something very inappropriate with another co-worker.

I am very good friends with our ES staff and our facilities management staff and none of them have ever made me feel like that (all male).

Trust your feelings.