On the note of why naked breasts wouldn't arouse you if you were a post-pubescent male--and I'm not referring to Ansdr's problem--there are guys who don't like certain types of breasts, especially saggy ones. (That is, unless you're in certain parts of the world. I think Africa has areas where women actually hang heavy rocks on their breasts to try to get them to sag, because breasts that have been used to feed a baby and aged are seen as more attractive.)

I think you can become numb to things. I don't know how to explain it, but even if you don't view those things a lot, it can happen.