Your not alone, this is just another side effect of children scattered to winds. It's more normal than we would care to believe to be attracted to our own.
My heart breaks for your future. My late 30's hasn't left me any wiser to advise you here.
My situation stAnds that eventually we went our separate ways, we wanted families of our own.
But every now and then he still turns up in the middle of the night with that sheepish look upon his face.

Talk to her, don't lock each other out, just be careful.
If you want to PM me feel free, even if it's just for someone to hear you out.