The dividing line between the two diagnosis is really whether psychotic symptoms are present when there's no mood component.

In your position, I'd ask your pdoc specifically about that issue and if that's why he thinks you're bipolar.