It's not inherently "terrible" or "bad" to not feel empathy. It just is. So you don't feel anything about this. Does it matter? Not really. I don't think it's reason to be confused or to hate yourself over.

Try to give yourself a break, it's okay.