I'm guessing here, but that guess would be that you the family you grew up with had no empathy either & you were never expressed those feelings to when you were young so that you could even know what they feel like.

We can learn that certain emotions exist, but learning to feel them is different than knowing that they exist......certain emotions /feelings like that are learned by actually living around them & experiencing them from others.

It's not right to get angry at yourself for not being able to feel something you never learned how to feel growing up.....

When you come upon things like this, it's a good thing to do a chain analysis into your earlier years of life to see where the problems have come from.....there is usually a good reason.