You can either cut out an abusive person from your life, or you can keep them around to abuse you some more. It's that simple. Nothing more complicated than that. Which causes you more anxiety, cutting him off, or having him continue to be enraged? I would think the "enraged" part would be more anxiety producing.

You can also attempt to speak to him about his behavior, but do it in a safe constructive way and refrain from saying anything that could turn it into a fight.
Set boundaries and ground rules. "If you continue to do A, then B" "If you continue to become enraged when I try to speak to you about a certain topic, then you will no longer be welcome in my life."