I have heard of men who react that way when they find out a woman they've been interested in is a survivor of SA. He may or may not be aware that it isn't her fault, but (spoilered for the imagery)
It's not fair, but I believe that's the way they're figuring it. And I think it's a shame. People CAN heal, if given a chance.
One very disgusting man complained about knowing so many women who were survivors of SA. He said (spoilered because this is really vile)