This very same thing happened to me. I was married young and realized I didn't love him and he didn't love me so we divorced after 9 years. I was single for 8 years. Then met my soulmate. We felt like we were meant to be together. Once when we were getting intimate, I called him by my first ex's name...I thought he would be mad but he wasn't. I cried because I thought I had hurt him and he'd think I still wanted ex. He thought it was no big deal. My therapist said it was like a flashback to my first real relationship. It didn't mean I wanted ex, but my brain just produced the wrong name.

There's nothing wrong with you. It happens when you have had former lovers. It does not mean you don't love him. Forget it and move on.