I have agoraphobia as well and have the same problem with the phone, although I am not so sure if mine is at the level as yours is. My phone fear kicks in when I have to answer the phone or someone asks me to make a phone call and I am unprepared> Foe example, something as small as my husband asking me to call and order dinner throws me into a panic. Semi related- I panic when the door bell rings or someone knocks on the door- I think that stems from a fear of "what if" which ties into my paranoia.
On a good note, I have been able to find therapists on line where you can simply send an email to make an appointment and if you let them know you are agoraphobic, some make housecalls.
Hope this helps, thanks for sharing about the phone, it makes me feel like I am not alone.