You are shaking because deep down you are worried about being punished for standing up for yourself instead of continuing the codependent behavior pattern he keeps pulling you into. I agree with other posters in that you will need to work on pulling away from your old habit of giving into codependency. You are not letting him down, you are letting yourself down if you go back for more of his abusive behaviors.