In reading the literature, the language suggests that there could be such an effect. I'd have to read the full-text journal articles, to be sure. What I would say, without that complete information, is that the most sensitive individuals may experience symptomatic disturbance in testosterone levels. But generally, they're saying it doesn't happen.
Lar