It sounds to me like it's possible that the doctor is right, but my experience of bipolar disorders is limited.

The one suggestion I would make, and this is based on your obvious intelligence and willingness to find out what's wrong is to take your time to find the answer. I know it's frustrating but taking the right course of action will help you massively in the long run. Even if it means taking the meds for now, which you clearly don't want to do. Even if you don't have a bipolar disorder, if the lithium is helping the symptoms it could give you the headspace to concentrate on finding the right answer.

Good luck