1) He's cheated on every woman he's ever been with
2) He gets mad at you when you bring it up he says "get over it" but he's still in contact with the woman with whom he had the affair (since they are in grad school together). Can you trust that it's really over?
3) When you fight he says he hates you and then asks what YOU are doing to make HIM stay. Shouldn't he be asking himself what He is doing to make YOU stay?
4) He dumped you so he could see the woman he had an affair with. Then he got back together with you and kept seeing her. Then slept with her the week you got married? What was he doing? Just making sure he was making the right choice??
5) Your words, "I'm fighting to keep loving him." Should you have to fight?

Cut and run, girl. You know the saying "a leopard doesn't change it's spots"? IMHO this guy of yours is a leopard.

It may have been a while ago, but that is the sort of thing that can shake a relationship to the core, and major repair work needs to be done. Has he been willing to do the work? Sure, at some point you should be able to trust again, but that's not going to happen without doing the work.

I'd like to say, give it time, do marriage counseling, but reading this story makes me think things are just not going to change.