Seriously, nothing is wrong with this at all. Let's face it, our underwear's just so much more fun and sexy than men's!
It's harmless, unless it's interfering with your life. Hopefully with time, and communication, your wife will be able to get past this little quirk of yours. Enjoy!