I guess it can be a toss up. How do you think your memory would be if you were unmedicated and stable, depressed or manic? Sometimes meds are the least worse option, at least they are for me unless I am stable for a long period of time...which rarely happens.

Episodes themselves mess with our memory too. It takes me months to get my brain back after a bad episode. This is a horrible position for you to have to be in. Are you stable enough to try going off the most likely culprit and see what happens? i wish you all the best.