Abuse is not an isolated incident. It's part of a pattern. From what I can tell in the OP, nothing happened only once. It happened again and again, in different forms. No matter what color you paint a pile of dog poop, it's still dog poop.

Yes, if you were knocked around "only a few times," and underlying it all is emotional abuse and manipulation, then I'd say you were physically abused as well.