To heck with weening. Go cold turkey. Don't ask this guy out again. Refuse if he asks you out. You can do it nicely, but refuse. I think you probably "can be yourself" with him because you know in your heart of hearts he isn't serious about you. I'd make myself scarce on those days he had to be at my workplace for good measure.