I don't hear what you're describing as abusive. My pdoc has a very small office and it's kind of dumpy. I'm not very comfortable in it, but I like him, so I go.

I like bpd2's advice. You should probably give him at least one more try. But don't discount your gut feelings, either. He may just be overly effusive and his style is not matching yours. If that's the case, then keep looking. To me, I think you just "know" when it's right.