This is what I mean. It may be warranted. It may not be. Same thing with the reasoning 'He called you a slut, you have to leave him, no matter what'-line.

Still, I do agree that this doesn't sound like a very good relationship. It is not like he is calmly explaining to you his very real but self-admitted faulty feelings. It is not like he is telling you he thinks he has these feelings that are unfair to you. It is not like he admits he feels your past reflects a 'slutty past version of yourself' he has trouble with accepting.

If he brings this up whenever you have a disagreement, if he blames you for his weaknesses. If he truly insults you and acts hypocritically, not concerned for your feelings in all this, then he is being wrong, clearly.