Holding you down, while he is mad?! That is being physically abusive and it's called restraining you and not allowing you to walk away from the argument. Which, you have a right to walk away from any argument.

He's in and out of your life. Sounds like a long, rocky, possibly futuristically dangerous road to be on.

What keeps you staying with him?