You aren't being oversensitive. You just care about your job and company a lot. You care about what happens, probably more than people with even greater responsibility.

Are there any small steps you could take to improve the situation? Could you tell people who are looking to you to sort out the chaos that they should talk to someone else?

Also, you say that she has most of your job. Are you worried that she is going to steal all of your job and leave you with nothing? Would it be possible to get a completely different job within your company, one that leaves you less exposed to this person?