All I can think of, is that it will get better with age. It's a matter of trusting that you bring something unique and beautiful to the present, that clearly didn't exist from his past.
Trust me, men like the new women better than the old lovers, and usually, for women, it's mutual.
Keep your chin up.