It's only a problem if you aren't able to work, be social and go about your daily life. As long as it doesn't dominate the aspects of your day-to-day situations, there isn't really anything wrong with it. I have the same interest (I wouldn't even call it a fetish), except I like full-back panties. I've have it since high school and I am 27 now. You're doing fine, there's nothing to worry about and I hope you can be at a place where this simple turn-on is comfortable and does not bring anxiousness.