would you get anxious by being around maybe just a few people who you gradually got to know. I thought maybe volunteering reading to a child that you might see each week would be fun.
Sounds like you want some social interaction -- just not alot? or none at all from no one.

Perhaps you could advertise to pet sit. Then its not a person but someone you can care for during the day. I need a cheap pet sitter myself!!!!

hmmmmmm. If you wanted a job.. housecleaning would be good because usually the people aren't there and it gives you something to do.

Maybe there are support groups for agoraphobics or people with depression. That way they would be sensitive to your issue and you could commensurate about similar issues sorta like psychcentral but in real life..

just some ideas