My therapist always says to do the thing that makes you most anxious so you can learn to work through it. I know it's not the answer people with anxiety want to hear (I hate when she says it lol) because we are really good at not doing things that make us anxious. I would suggest staying with the job and trying to work through it. I hope you are working with a therapist so you can figure out what makes you so anxious.