I agree with the others that you are not wrong. But I will say, based on my own experiences, it's best to be clear about your feelings about it with your current boyfriend if it's ever brought up.

I hadn't fully moved on from someone I've been with for 5 years while I dated two more people and when asked if I still had older feelings I always denied them. All that did was make things more tense and uncomfortable if I would have just been honest and said I wouldn't go back, but I wasn't over it. If the other person really cares for you they'll get past it, with you.