They call previous employer unless you indicate “do not call my employer”. I know people do ask for that if they still on their job, they don’t want their boss to know that they are looking. It’s common to ask for it.

But with a former employer they’d just call to confirm if you actually worked there. Not for references per se but to confirm the dates. I don’t think they can demand some kind of disclosure about quality of your work. Anyone can say they worked in The White House or Pentagon. Prospective employers call to confirm (if your job isn’t a public knowledge type of job)