Well it depends, how did your mother and sister treat you.

Did they ignore you. One of the worse things you can do is ignore someone totally. Maybe even tho he was abusive he gave you attention?