My opinion is you shouldn't feel any guilt about not sympathizing. Don't believe there is a flaw in you because you feel this way. You should decide your attitude towards him. I'm sure your family would want you to forgive, but that is because it will make them feel better. You were the one hurt. Be compassionate to yourself and to people you want to feel compassionate towards.