You never have to take medication. It's your choice. Even if the attacks aren't 100% severe they can still be a bother and I believe most attacks that people have will be more "anxiety" than they are "panic."

And yes, I think the spurts of anger can result from a similar process to the anxiety.