I have to say that i am in exactly the same boat as you. I lost nearly half my body weight and practically all my breasts.
My fiance used to say that the thing he loved most about me was my breasts so im sure you can imagine how devastated I was when I actually came to the realisation that they were no longer as "perky" and big as they had been before the weight loss.
I also went through the fears of him taking an interest in a women who had bigger breasts then i did, but at the end of the day if something as simple as breasts could make a man stray then he wasnt yours to begin with.

If YOU accept and love your body, trust me honey....he will too.