There’s a famous quote, usually wrongly attributed to Keynes that ‘it’s better to be vaguely right than precisely wrong.’ In a new book “noise” refers to inconsistent decisions, that need not be biased in any consistent way. But consistency is not the only value that matters. Academics are sometimes evaluated on the basis of the number of articles they publish. If this is done conscientiously, then the evaluation is consistent, and in that sense “fair.” But maybe there are other criteria that are harder to measure, but that matter more, like the profundity and insight of what is published. Evaluating on the basis of well-measured criteria, that matter less, rather than poorly-measured criteria, that matter more, may increase unfairness in a deeper sense.
(p. 10) A study at an oncology center found that the diagnostic accuracy of melanomas was only 64 percent, meaning that doctors misdiagnosed melanomas in one of every three lesions.
When two psychiatrists conducted independent reviews of 426 patients in state hospitals, they came to the equivalent of a tossup: agreement 50 percent of the time on what kind of mental illness was present.
. . .
Doctors are more likely to order cancer screenings for patients they see early in the morning than late in the afternoon.
. . .
In a study of the effectiveness of putting calorie counts on menu items, consumers were more likely to make lower-calorie choices if the labels were placed to the left of the food item rather than the right.
“When calories are on the left, consumers receive that information first and evidently think ‘a lot of calories!’ or ‘not so many calories!’ before they see the item,” Daniel Kahneman, Olivier Sibony and Cass R. Sunstein explain in this tour de force of scholarship and clear writing. “By contrast, when people see the food item first, they apparently think ‘delicious!’ or ‘not so great!’ before they see the calorie label. Here again, their initial reaction greatly affects their choices.” This hypothesis is supported, the authors write in a typically clever aside, by the “finding that for Hebrew speakers, who read right to left, the calorie label has a significantly larger impact if it is on the right rather than the left.”
These inconsistencies are all about noise, which Kahneman, Sibony and Sunstein define as “unwanted variability in judgments.”
. . .
As the authors explain in their introduction, a team of target shooters whose shots always fall to the right of the bull’s-eye is exhibiting a bias, as is a judge who always sentences Black people more harshly. That’s bad, but at least they are consistent, which means the biases can be identified and corrected. But another team whose shots are scattered in different directions away from the target is shooting noisily, and that’s harder to correct. A third team whose shots all go to the left of the bull’s-eye but are scattered high and low is both biased and noisy.
Despite its prominence in so many realms of human judgment, the authors note that “noise is rarely recognized,” let alone counteracted. Which is why the parade of noise examples that the authors provide are so compelling, and why gathering the examples in one place to demonstrate the cost of noise and then suggesting noise reduction techniques, or “decision hygiene,” makes this book so important. We are living in a moment of rampant polarization and distrust in the fundamental institutions that underpin civil society. Eradicating the noise that leads to random, unfair decisions will help us regain trust in one another.
“Noise” seems certain to make a mark by calling attention to the problem and providing a tangible guide to reducing it. Despite the authors’ intimidating academic credentials, they take pains to explain, even with welcome redundancy, their various categories of noise, the experiments and formulas that they introduce, as well as their conclusions and solutions.
For the full review, see:
Steven Brill. “No Chance.” The New York Times Book Review (Sunday, May 30, 2021): 10.
(Note: ellipses added.)
(Note: the online version of the review has the date May 18, 2021, and has the title “For a Fairer World, It’s Necessary First to Cut Through the ‘Noise’.”)
The book under review is:
Kahneman, Daniel, Olivier Sibony, and Cass R. Sunstein. Noise: A Flaw in Human Judgment. New York: Little, Brown Spark, 2021.