I think you hit the nail on the head. This goes back to your childhood. It isn't so much THIS guy, but any guy that shows you attention. It could have been any guy that you had gone out with and had fun with. It still would have gone back to your childhood.
I suggest that you get into therapy and work on this issue and perhaps others. It's important to take care of this so you CAN have healthy relationships and not "scare" guys away with neediness. I can certainly understand it, as I was totally ignored as a child. I was in therapy for years (I don't think YOU"D need it that long) and had my issues taken care of. I'm so glad I did, as otherwise I would have been alone.
I wish you the very best. Please take good care of yourself, and let us know what happens, okay? God bless and Happy Holidays.

Hugs, Lee