I sort of relate. I certainly get rusty over longer periods of time. Short term, it's having the time to let the anxieties rear their ugly head again -- bad habits and negative thought patterns. Sometimes it passes rapidly if I cope well and people are nice enough. Sometimes it goes poorly and then I'll usually feel bad about it for awhile.
So for me it's just the social anxiety and feelings of inadequacy playing their usual head games. Therefore, it needs to be treated that way.
It isn't my depression as such. Although the depression says "Oh, why even bother?"
I try to remind the depression that I don't recall asking for its opinion.