Good question. I think it's mainly that I keep thinking about "what-ifs":

-What if I get the wrong number?
-What if someone else answers?
-What if they're not in, and I have to leave a message (either with someone else in the house, or an answerphone)? What if I give the message wrong?
-What if I forget something, and have to call back?
-What if they can't understand me? (Sometimes I mix words up or talk too fast if I get nervous. )
-What if I interrupt them in the middle of something?

There are probably more, but those are the ones I can think of at the moment. With mobile phones it's a bit easier, because I know who's going to answer, but then there's problems of losing the signal suddenly, plus if I make the call somewhere where it's quite loud (such as a shopping centre), I might not be able to hear them.

There's also the fact that I'm relying solely on using my voice. You can't use facial expressions on the phone, and you can't read the other person's, either. If I'm talking to someone in person, I can make a face if I say something stupid, I can smile and nod when the other person is talking, I can sometimes point at whatever I'm talking about...you can't do that with phones! It feels to me like it's more easy to be misunderstood.

Hope I'm making sense, sometimes I'm not sure myself why I hate making phone calls so much...