Selfy...I'm age 57, and been thru this kind of thing a number of times, both before and after my 20 year marriage. I think it has a lot to do with being vulnerable, being a pleaser, and, as you say, not wanting to let go of something that feels familiar and safe.
But, please, don't encourage this fella, especially if you want to be free of it. Also, be careful of accommodating someone who could become a stalker. There is a fine line there. I know from experience.
If he won't leave you alone, despite your expressions to be left alone, after which you succumb to his demand, and say "okay," then you need to put your foot down, and say NO!
Patty