Yes, admin work can be done in a number of environments and industries. So keep in mind the industry you are looking at, take note of the environment when you interview there and ask what the company culture is like.

Sounds like you need to steer clear of sales and recruiting industries, or of any industry that requires outgoing people and socializing. Perhaps look into being in an industry that caters more to the introvert. I worked as an admin in an academic environment, for example, and that seemed to be a bit more introverted. (((Hugs))))