You're not stupid at all Hippie.
Our anxieties aren't always logical, but I think that can be the nature of fear. Like with other phobias, my mum is scared of spiders. But when you think about it, a spider is so small, they can't bite you (in this country at least :P) or do anything to you... but when she sees one she's terrified.
You're anxious about driving and that's ok! You can overcome it, and you've proved you can because you drove all those miles the other day. Today is a bad day for driving, we all have days when our anxiety is higher. But you'll have another day when it's lower and you'll be able to drive that day.

Are you sure you can't talk to your family or whoever you're living with? Why is it that you HAVE to drive? Are there buses, taxis? You know what, actually, my mum doesn't like driving. It's not because she suffers with anxiety, she just hates driving, she doesn't feel like she's good at it, she hates having to park or reverse and things like that. About 10 years ago she decided to stop driving. My step dad still drives so that covers holidays or days out. And she gets the bus to work.
Do you absolutely HAVE to drive?
If you do you can overcome it. But if you can't, can you just do something else that day or get public transport? Maybe you're putting more pressure on yourself to drive than you need to?