You need to figure out what you want in life, I know it is difficult as you are young, but you have to teach people how to treat you.
Your father is only human and is not perfect, he was abusive but you are an adult now, and it is your choice to see him, and under your terms. If you need to tell him how you feel you can write him a letter, and be done with it.
If you don't want a relationship with the man, break it off, if he dismisses what you say, that is not a good partner. So be done with it, if that's what you want. Draw a line in the sand, and say I am done, and stick to it, don't keep contact.
Put yourself first, if they aren't meeting your needs, don't pursue it.