Could it be that knowing your father was rejected by his father which as you share led to your father's early death, that you wanted something different. What you have is a man who won't or isn't rejecting you, he begs you to stay and he does provide for you "except" when it comes to sexually where he sounds like he is sexually selfish. Also, I am wondering if your H is like your father in that he might have low self esteem and gives in to you and deep down you know that if there is going to be any kind of breakup it will be "your" decision.

It just seems from what you share that you have relationships where "you" can disconnect and choose to distance in a knowing that the other person will still be around if you decide to engage them. Almost as though you like the loyalty, even if the other person is "selfish". I think a part of you actually likes to know this so called friend would like to have what you have too because in some way it helps you to see some "value" even if it's not completely filling "all" your needs.

How old were you when your father died?