Hi mulan. I can see that you have common ground with this person so it makes sense you would enjoy their company.
The biggest problem I see is that if you did get involved with a coworker and a relationship developed, then there is the problem of what would happen if the relationship went bad. Would that mean one or both of you might have to leave the company.
You said " He had shown affection towards me, but if he likes me, he would make a move..." I agree and he may have thought out the possible consequences of a relationship with a co worker and that could be why they are not making a move. He sounds very polite and considerate.
It seems like you are fortunate to have a good coworker.