I honestly don't consider Zoloft one of the scarier drugs. Did you doctor tell you to start with a half dose for a week and then go up to the full 50mg? You should do that. It will minimize the initial shock to your system, which might produce the more painful side effects. Mostly on Zoloft I was insomnia and restlessness for a few days, then headaches and tiredness for some weeks after that. If you give your body time to gradually get used to it, you will have the best response.

Benzos (what Xanax is) can cause dependence and could result in you needing to take a higher dose to get the needed reaction. If you feel you really need sometime to control your anxiety or panic attacks, go with the Zoloft since it will be constantly acting in your system, not as need like the Xanax. Alternatively, you could take the Xanax as needed and work on non-drug techniques for controlling your panic attacks. You can get into therapy for that, try meditation, etc. (or some combo of med. and this).

A year ago, Zoloft was the first antidepressant I took. I was scared of it as well, but I took it because I felt things could only get better on it.