Hester, hunny... You say, "if he is a sociopath, i need to cut it off, now." No...you need to cut it off, period! What you describe is a man who's very good at getting women addicted to him, and then controlling them! A counselor once described this to me (as I was involved with a similar) as I was completely absorbed and obsessed, waiting, waiting to hear from him, see him. The counselor said the intermittent contact, i.e., seeing him, hearing from him thru email or phone call, was much more addictive than the constant comfort of security, of knowing when and where. And he likened it to studies with animals, mice, for instance, who keep coming back for the intermittent reward...they're addicted! This man is so good at this, he's gotten you to behave in ways that make you question your sanity, your values, who you are! As you said, he has treated other women the same, and they still remain in contact (sending presents, ugh).
In you, he zeroed in on your vulnerabilities, because of the nature of your marriage, even your age. When you get some distance on this situation, completely away from him for a substantial period of time, you will be appalled.

Now...I'm going to tell you what you should do...Just stop it. You don't even need to give him an explanation. Talking and reasoning will just diminish your power and self worth. Because I suspect you are under his control (based upon what you've described here), you could have a very hard time letting go of this. If you can find counseling help, it would probably be a good step. And, oh, by the way, his ED is not your problem, it's his. You can't save him. Save yourself.