Sounds like you've gotten to sort of need him -- for reassurance or something.

If he weren't into reassuring you, or he felt leaned on and didn't want to feel that way, I can see where he might have a bit of a problem with you needing him and might feel like keeping more of a distance than you wanted.

I can see where trying to hold on to him (assuming for the moment that that's what you're doing) could have two results that you might not want: it could make him uncomfortable (and push him away), and it could make you feel as if you weren't in control of how the relationship went and what you could do for yourself.

If you could bring yourself to let him to go his way, whether that way was going to lead past your door or not, you'd at least be putting yourself back in charge of how you felt. Of course your being willing to let him go wouldn't make him want to be with you more -- that'll always be up to him -- but it just might take some pressure off him and let him be more comfortable around you, if he ever did feel like stopping by.

__________________________
If you love something, let it go.
If it comes back, it's yours forever.
If not, hunt it down and kill it...
(No, no, not that version, the other one! LOL)