(p. C12) In the early 1980s, Paul Sackett, a psychologist at the University of Minnesota, began measuring the speed of cashiers at supermarkets. Workers were told to scan a few dozen items as quickly as possible while a scientist timed them. Not surprisingly, some cashiers were much faster than others.
But Mr. Sackett realized that this assessment, which lasted just a few minutes, wasn’t the only way to measure cashier performance. Electronic scanners, then new in supermarkets, could automatically record the pace of cashiers for long stretches of time. After analyzing this data, it once again became clear that levels of productivity varied greatly.
Mr. Sackett had assumed that these separate measurements would generate similar rankings. Those cashiers who were fastest in the short test should also be the fastest over the long term. But instead he found a surprisingly weak correlation between the rankings, leading him to distinguish between two types of personal assessment. One measures “maximum performance”: People who know they’re being tested are highly motivated and focused, just like those cashiers scanning a few items while being timed.
The other type measures “typical performance”–measured over long periods of time, as when Mr. Sackett recorded the speed of cashiers who didn’t know they were being watched. In this sort of test, character traits that have nothing to do with maximum performance begin to influence the outcome. Cashiers with speedy hands won’t have fast overall times if they take lots of breaks.
. . .
The problem, of course, is that students don’t reveal their levels of grit while taking a brief test. Grit can only be assessed by tracking typical performance for an extended period. Do people persevere, even in the face of difficulty? How do they act when no one else is watching? Such traits often matter more than raw talent. We hear about them in letters of recommendation, but hard numbers take priority.
The larger lesson is that we’ve built our society around tests of performance that fail to predict what really matters: what happens once the test is over.
For the full commentary, see:
JONAH LEHRER. “Measurements That Mislead; From the SAT to the NFL, the problem with short-term tests.” The Wall Street Journal (Sat., APRIL 2, 2011): C12.
(Note: ellipsis added.)
The classic article correlating maximum and typical performance, is:
Sackett, Paul R., Sheldon Zedeck, and Larry Fogli. “Relationships between Measures of Typical and Maximum Performance.” Journal of Applied Psychology 73 (1988): 482-86.