What type of medication are you taking for your anxiety? If it is a "take as needed" the effects are nearly immediate but are short lasting, 6 hrs max, usually less. If it is on a regular schedule it is most likely a medication that builds up in your system over time. These generally take 3 or more weeks to hit therapeutic levels, so in that case waiting would make more sense.

It's possible you are getting worse, but sometimes something triggers panic attacks out of nowhere. With me, they increase when I am under a lot of stress and I can go several months, I've even gone years, without them when I'm not under as much stress. Hormones can also effect them.