I'm sorry about your situation. Like other suggested, I think the best solution is to slowly cut contacts with him.. trying to re-build anything would probably just hurt you more in the long run.