There have been studies showing (and my pdocs have all agreed) that some people who have bipolar have cognitive problems during all phases of the illness.

I was severely depressed for nearly a year and never recovered. I've gotten a little better, but I'm certain that some of it is permanent. My docs think I have MS, though, so that might be part of it...

EDIT: Here's a link to a new-ish study about cognitive decline in schizophrenia and bipolar. They found that 31% of the bipolar patients in their study experienced severe intellectual decline. I'd assume that a significant proportion also experienced a mild to moderate decline. It's definitely permanent for some people.
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4423162/

And this is from an abstract of another article: " The latest research data suggest that the neurocognitive impairment is an enduring component of BD and represent a core primary characteristic of the illness, rather than being secondary to the mood state or medication." - http://link.springer.com/chapter/10....-642-37216-2_4