I think there is no rules to forbid you to be friends or lovers with your family doctor or another doctors. Why not?
Is he married? Is he free?
I don't know how to tell him, maybe write him email or write him on facebook, I really don't know. Invite him to the cafe maybe. I think it would be hard to talk to him when he's working esspecially if you see him for 5-10 min.