I used to feel the same way. My hubby convinced me that he loves to do it, and every single male friend I have talked to had said the same. Trust me, if he didn't want to do it, and you told him you didn't want it, he would be soooo happy to hear that. For many men, making a woman feel good is a huge ego boost... And, of course, it really does a lot for them, sexually, to be the one delivering the pleasure. Your man is not going to be "grossed out", I promise!
Again, I used to think the same thing. I was so opposed to the idea, for a number of reasons. But if your man wants to go down, LET him. And ENJOY. And don't be self-conscious, or wonder what he is thinking. How many men would look at a woman's v and think, "OMG, gross, I am not touching that?" Hehe. As much as he loves oral sex, you will too, as long as you are relaxed and ready to let him have a turn pleasuring YOU.