Everyone has a past and that guy could not stomach it.

And no, the damage was not done because you didn't get help immediately. The damage was done because you were with someone insecure, controlling and violent. That was never going to work.

He is judging you for your past, *not* the person you are/were with him. How on earth is that *yout* fault?