unfortunately great sex with a sexy guy does not = good relationship.

Probably best to move on, it sounds like he may have intimacy issues, or maybe just using you. Maybe I'm wrong, I don't know him, but if he is a trigger for you, then LET HIM GO!