I never said he doesn't run his business right. I told him that strong leadership has been lacking within my department specifically, that they are under-staffed and therefore, that I became overworked and burnt out. That's very different from telling him that he runs his business poorly. I told the truth, and he already is aware of these things, yet doesn't do anything or act quickly enough to fix them so that good people don't leave. That's his issue and his problem to resolve. I also said many positive things about the company, about him and about my experience there.
And I do worry about getting a good reference. I need a good reference, as the company owner before him won't give me one because she bullied me, and I got her into trouble with HR at the company. I cannot keep relying on references from 7-10 years ago. I think in my case, my concerns are very valid.