I think you are forgetting your half of the responsibility; he did not just marry you, you also decided to marry him. If he wasn't letting go and working with the relationship in its current state, you probably should not have married him (yet)?

If someone were constantly discussing my previous behavior, I would find that annoying at a minimum and scary, as you say, otherwise. You are in charge of your behavior; he can decide to forgive and move on or not forgive/move on but you always get to decide what you will do.

Decide what you want, not based on what he does; you have no control whatsoever on his behavior! Tell him you are tired of discussing old behavior (of anyone's!) and if he does not figure out how to shift his attitude to here and now or get help for it, you will leave him and then do that or, you can stay and nothing will change and you will get depressed and more unhappy than you are now.