Wow. What a sad story. I hope you're as angry at your dad as I am. And the first huy you mentioned. And the second one too.
It's a tough situation and I hope someone wiser than I am is thinking about how to answer this. I think that our parents start us on the path to self worth and self esteem.
Apparently yours didn't. That leaves you. How do you re-parent yourself?
First thing I would suggest is to get out of this abusive relationship. There doesn't seem much point in trying to reprogram yourself and build your self esteem if there is someone there who is constantly undoing your work. It's going to be hard enough under good circumstances.
That's where I'd start at least.