I would get a new pdoc. See if he can do some testing on you. Yes I wasn't diagnosed until I was 36. I had my first manic kinda psychotic episode when I was 21 though after I gave birth to my son. My primary drs always said they thought I had bp but I didn't believe them. When I was 36 I went into a bad mania that turned into a psychosis and ended up in the hospital and was diagnosed with bipolar disorder 1 and PTSD. But I also took like 4 different tests in the hospital. After seeing it in black and white I completely believe it now.
I think stay on the lithium. For me I couldn't imagine taking any adhd meds. I'd be completely manic. But I don't have adhd at all. I really think seeing a new pdoc that will take time for you is your best way to go. I do take anti depressants, but only the ssris. The snris make me go mixed. But I also can take them bc I'm on a high dose of a mood stabilizer and a strong anti psychotic. But your manic episode could have been anti depressant induced. I would hope for that and hope you're not bp. It's no fun. Good luck.