I know it's scary, but maybe you can let him in on the fetish in a gradual way. Maybe you could start by saying that you find white underwear more appealing than any other color. That doesn't seem too weird. If you can get to a point where he wants to participate in your fetish, it might lead to a healthier relationship.