I think he's using you too, you need to stop giving him your world if he doesn't want anything but sex. Eventually there will come a point where you'll have to say why am i here doing this with him? It will go on as long as you let him because he's getting the milk for free.