I want to be positive and stuff, I am just not in a place to write a lot about relationships.
Except to say a few things, it doesn't seem like this friendship is healthy for you, and you know it, I admire your honesty. I'd think it wise to cut ties with this man.
Secondly, I hear and maybe it's my bad hearing, but I hear you put him up here, and you down there. Why you are not good enough. What I would ask is, is he good for you. Does he have what YOU need.
I'm sorry I wish I had more, it's a very hard place to be