Disliking that phrase sounds like a healthy reaction to stereotyping. (I mean, I don't know the context where you hear it, but I can't actually imagine any context where it would not sound really inappropriate to me. Unless one is talking to one of the Village People or something.)