Actually, it probably has everything to do with your father. Our relationships to our family is actually a huge part of how we define ourselves. The key is to examine those relationships and learn what we unconsciously think about ourselves and then try to find a way to redefine it. I had a similar issue with my father until I took a close look at my relationship with him. Once I got beyond the stuff that made me feel worthless and angry, I was able to see how he shared his love with us. I was expecting him to be something he wasn't, so it completely ruined how I was viewing him. Once I saw how he loved me (not in a the way I was expecting him to), my entire perception of him changed. And when that changed, half of my issues changed with it.