I understood what you said and what you mean. You're definitely in "self-destruct mode" and you know it. What are you afraid of if you kick this man out of your life?
Some elements of what you may be afraid of already exist. He doesn't care for you the way you want to be cared for. You won't be missing that.
You are "settling for second best" which hurts you. Why not be "not at all"? You'd be happier with yourself.
Come out of the shadows, Hon. Make your own light! You can do it! It's really tough, but it is NOT impossible. We're here to help you through.
__________________
Psalm 119:105 Thy word is a lamp unto my feet, and a light unto my path.