Lovedeeply,
It sounds like you have a wonderful husband. The fact that you can talk with him about your ex makes him sound even more appealing
If you ex ended up perturbed with you, perhaps that speaks to how a longer term relationship would have ended up with him. There is always the appeal of "what might have been" or just the thought of multiple people being amorous of you at once, but a man that will stick by you is worth his weight in the GQ gold you mentioned.
Good luck!