(p. B1) Environmental regulators in Gujarat, one of India’s fastest-growing industrial states, found themselves in an implausible situation a few years ago: Every single city breached national air quality standards. And yet environmental audits kept finding that factories met pollution limits.
So the Gujaratis hired some researchers from Harvard and the Massachusetts Institute of Technology to carry out an experiment, changing the way the audits were made. Instead of hiring their own auditors, companies had auditors assigned to them randomly. Instead of being paid by the companies they audited, auditors drew a fixed fee from a pool that all companies paid into.
Measured compliance rates abruptly plummeted. But once the new system was in place, the real emissions from polluting factories finally started to decline. The Gujaratis kept the new approach.
“When fact-checking is not done in an independent way, there is a long history of things turning out the way the entity being fact checked wants them to turn out,” said Michael Greenstone of the University of Chicago, a former chief economist for President Obama’s Council of Economic Advisers who was one of the researchers involved in the study. “Until you change the incentives, this will not change.”
The problem may seem remote, but it turns out that the same incentives apply in the United States, even in programs that, at first glance, appear to provide an unmitigated benefit.
Last month, the Energy Department released an extensive report assessing the impact of the federal weatherization program, which was begun in 1976 to shield the homes of low-income Americans from the elements, save them money on heating bills and improve energy efficiency.
It concluded that weatheriza-(p. B10)tion — insulating homes, changing boilers, plugging leaky windows and the like — was a stellar investment. Not only were the energy savings substantially larger than the cost of weatherizing homes, the report found, but the gains soared even more once the broader impacts on health were taken into account.
“The results demonstrate that weatherization provides cost-effective energy savings and health and safety benefits to American families,” the Energy Department announced.
But do they? When Professor Greenstone and two other independent economists looked under the hood — not a trivial challenge, given the report’s 4,500 pages — they found a collection of idiosyncratic choices and unorthodox assumptions that severely undermined the credibility of the enterprise.
In the end, they concluded, the government research effort, which was led by the Energy Department’s own Oak Ridge National Laboratory, cannot tell us whether weatherization is a fabulous program or a waste of taxpayer dollars.
For the full commentary, see:
Eduardo Porter. “ECONOMIC SCENE; For Government That Works, Call In the Auditors.” The New York Times (Weds., OCT. 7, 2015): B1 & B10.
(Note: the online version of the commentary has the date OCT. 6, 2015, and the title “ECONOMIC SCENE; For Government That Works, Call In the Auditors.”)