I say I don't care a lot too. I'm on the Autism spectrum so when someone asks me something it's really hard for me to think of something else besides I don't care because I usually don't have an opinion. But I've been told numerous times that its rude. I try not to say it. Its just hard.