Category: Methodology

Kenneth Arrow Had Broad Knowledge Beyond Economics

(p. A21) Professor Arrow was widely hailed as a polymath, possessing prodigious knowledge of subjects far removed from economics. Eric Maskin, a Harvard economist and fellow Nobel winner, told of a good-natured conspiracy waged by junior faculty to get the better of Professor Arrow, even if artificially. They all agreed to study the breeding habits of gray whales — a suitably abstruse topic — and gathered at an appointed date at a place where Professor Arrow would be sure to visit.
When, as expected, he showed up, they were talking out loud about the theory by a marine biologist — last name, Turner — which purported to explain how gray whales found the same breeding spot year after year. As Professor Maskin recounted the story, “Ken was silent,” and his junior colleagues amused themselves that they had for once bested their formidable professor.
Well, not so fast.
Before leaving, Professor Arrow muttered, “But I thought that Turner’s theory was entirely discredited by Spencer, who showed that the hypothesized homing mechanism couldn’t possibly work.”

For the full obituary, see:
MICHAEL M. WEINSTEIN. “Kenneth Arrow, Influential Economist and Nobel Laureate, Is Dead at 95.” The New York Times (Weds., FEB. 22, 2017): A21.
(Note: the online version of the obituary has the date FEB. 21, 2017, and has the title “Kenneth Arrow, Nobel-Winning Economist Whose Influence Spanned Decades, Dies at 95.”)

Trump Victory Undercut Faith in Big Data

(p. B1) It was a rough night for number crunchers. And for the faith that people in every field — business, politics, sports and academia — have increasingly placed in the power of data.
Donald J. Trump’s victory ran counter to almost every major forecast — undercutting the belief that analyzing reams of data can accurately predict events. Voters demonstrated how much predictive analytics, and election forecasting in particular, remains a young science: . . .
. . .
(p. B5) This week’s failed election predictions suggest that the rush to exploit data may have outstripped the ability to recognize its limits.
. . .
Beyond election night, there are broader lessons that raise questions about the rush to embrace data-driven decision-making across the economy and society.
The enthusiasm for big data has been fueled by the success stories of Silicon Valley giants born on the internet, like Google, Amazon and Facebook. The digital powerhouses harvest vast amounts of user data using clever software for search, social networks and online commerce. Data is the fuel, and algorithms borrowed from the tool kit of artificial intelligence, notably machine learning, are the engine.
. . .
The danger, data experts say, lies in trusting the data analysis too much without grasping its limitations and the potentially flawed assumptions of the people who build predictive models.

For the full story, see:
STEVE LOHR and NATASHA SINGER. “How Data Failed Us in Calling an Election.” The New York Times (Sat., NOV. 10, 2016): B1 & B5.
(Note: ellipses added.)

Double-Blind Trials Are Not the Only Source of Sound Knowledge

(p. 1) . . . while all doctors agree about the importance of gauging the quality of evidence, many feel that a hierarchy of methods is simplistic. As the doctor Mark Tonelli has argued, distinct forms of knowledge can’t be judged by the same standards: what a patient prefers on the basis of personal experience; what a doctor thinks on the basis of clinical experience; and what clinical research has discovered — each of these is valuable in its own way. While scientists concur that randomized trials are ideal for evaluating the average effects of treatments, such precision isn’t necessary when the benefits are obvious or clear from other data.
Clinical expertise and rigorous evaluation also differ in their utility at different stages of scientific inquiry. For discovery and explanation, as the clinical epidemiologist Jan Vandenbroucke has argued, practitioners’ instincts, observations and case studies are most useful, whereas randomized controlled trials are least useful. Expertise and systematic evaluation are partners, not rivals.
Distrusting expertise makes it easy to confuse an absence of randomized evaluations with an absence of knowledge. And this leads to the false belief that knowledge of what works in social policy, education or fighting terrorism can come only from randomized evaluations. But by that logic (as a spoof scientific article claimed), we don’t know if parachutes really work because we have no randomized controlled trials of them.

For the full commentary, see:
PAGAN KENNEDY. “The Thin Gene.” The New York Times, SundayReview Section (Sun., NOV. 27, 2016): 1 & 6.
(Note: ellipsis added.)
(Note: the online version of the commentary has the date NOV. 25, 2016.)

The academic article calling for double-blind randomized trials to establish the efficacy of parachutes, is:
Smith, Gordon C. S., and Jill P. Pell. “Parachute Use to Prevent Death and Major Trauma Related to Gravitational Challenge: Systematic Review of Randomised Controlled Trials.” BMJ 327, no. 7429 (Dec. 18, 2003): 1459-61.

Science Can Learn Much from Outliers “Who Are Naturally Different”

(p. 1) Abby Solomon suffers from a one-in-a-billion genetic syndrome: After just about an hour without food, she begins to starve. She sleeps in snatches. In her dreams she gorges on French fries. But as soon as she wakes up and nibbles a few bites, she feels full, so she ends up consuming very few calories. At 5 feet 10 inches tall, she weighs 99 pounds.
Now 21 years old, she is one of the few people in the world to survive into adulthood with neonatal progeroid syndrome, a condition that results from damage to the FBN1 gene.
. . .
(p. 6) Dr. Chopra told me that, as far as medical science is concerned, Abby Solomon is worth thousands of the rest of us.
. . .
“Nothing comes close to starting with people who are naturally different,” he said. This is why he searches out patients at the extreme ends of the spectrum — those who are wired to weigh 80 pounds or 380 pounds. He said, “We have the opportunity to help a bigger swath of humanity when we learn from these outliers.”
In 2013, after hearing about Ms. Solomon’s unusual condition from another patient, he asked her to visit his clinic. Ms. Solomon warned him that she would be able to carry on a conversation for only 15 minutes before she needed to snack on chips or a cookie. That remark inspired a revelation. Dr. Chopra realized that “she had to eat small, sugary meals all day to stay alive, because her body was constantly running out of glucose,” he said.
The clue led Dr. Chopra and his colleagues to their discovery of the blood-sugar-regulating hormone, which they named asprosin. Ms. Solomon’s natural asprosin deficiency keeps her on the brink of starvation, but Dr. Chopra’s hope is that an artificial compound that blocks asprosin could be used as a treatment for obesity. He and his team have already tested such a compound on mice, and found that it can reverse insulin resistance and weight gain.

For the full commentary, see:
PAGAN KENNEDY. “The Thin Gene.” The New York Times, SundayReview Section (Sun., NOV. 27, 2016): 1 & 6.
(Note: ellipses added.)
(Note: the online version of the commentary has the date NOV. 25, 2016.)

Jane Jacobs Studied the “Mess of Everyday Life”

(p. C6) The decidedly unpredictable and unscientific mess of everyday life was the passion of the urban theorist Jane Jacobs. For her, studying the street and the city was the key to understanding how things work. Robert Kanigel’s “Eyes on the Street: The Life of Jane Jacobs” has taken a place on my bookshelf right next to Robert Caro’s landmark biography of her nemesis, Robert Moses.

For Bierut’s full book recommendations, see:
Michael Bierut. “12 Months of Reading.” The Wall Street Journal (Sat., December 10, 2016): C6.
(Note: the online version of the review has the date Dec. 7, 2016, and has the title “Michael Bierut on Jane Jacobs.”)

The book recommended, is:
Kanigel, Robert. Eyes on the Street: The Life of Jane Jacobs. New York: Alfred A. Knopf, 2016.

“Celebration of Big Data” Becomes “Funeral”

(p. B8) DENVER — At the governor’s mansion here on Friday [November 11, 2016], past the columned entryway and the French chandeliers, Emmy Ruiz placed a hand on the shoulder of a fellow Hillary Clinton operative. “It’s like we’re at a funeral,” said Ms. Ruiz, dressed — perhaps coincidentally — in black.
Just days after Donald J. Trump’s surprise presidential victory, the nation’s professional political forecasters and persuaders — the pollsters, the ad creators, the campaign strategists — gathered in Denver for their annual convention. It was supposed to be a celebration of big data and strategic wizardry for a multibillion-dollar industry that has spent nearly a century packaging political candidates.
Instead, the conference of the International Association of Political Consultants felt like a therapy session for a business in psychological free fall.

For the full story, see:
JULIE TURKEWITZ. “At Conference, Political Consultants Wonder Where They Went Wrong.” The New York Times (Tues., NOV. 15, 2016): B8.
(Note: bracketed date added.)
(Note: the online version of the story has the date NOV. 14, 2016.)

Economist Removed from Plane for Scribbling Math

The seatmate was wrong to think the scribbling was Arabic, but was right to be alarmed.

(p. A13) In May [2016], an Italian economist from the University of Pennsylvania was removed from an American Airlines flight in Philadelphia after his seatmate became alarmed, thinking that the math he was scribbling on a piece of paper was Arabic, The Washington Post reported.

For the full story, see:
CHRISTINE HAUSER. “American Airlines Orders 2 Muslim American Women Off a Long-Delayed Flight.” The New York Times (Sat., AUG. 5, 2016): A13.
(Note: bracketed year added.)
(Note: the online version of the story has the date AUG. 5, 2016, and has the title “2 Muslim American Women Ordered Off American Airlines Flight.”)

Many Discoveries Take a Long Time Because “No One Really Looked”

Periods are a strange phenomenon. We don’t know why humans have them, or, to look at it another way, why most other animals don’t. Scientists say only 1.5 percent of mammal species have periods, and most of those are primates like us. The ranks of the menstrually afflicted grew a little bit recently, as researchers learned that female spiny mice have periods, too. They shared their findings on the bioRxiv preprint server.
. . .
Why did it take scientists so long to notice that these curious creatures were part of the period posse? “The answer, as with many discoveries in science, is that no one really looked,” said Hayley Dickinson, a reproductive physiologist and long-time spiny rat researcher at the University of Monash. “Everyone knew that rodents didn’t menstruate.”

For the full story, see:
Nowogrodzki, Anna. “First Rodent Found with a Humanlike Menstrual Cycle.” Nature (Fri., June 10, 2016).
(Note: ellipsis added.)

The preprint of the research mentioned above is:
Bellofiore, Nadia, Stacey J. Ellery, Jared Mamrot, David W. Walker, Peter Temple-Smith, and Hayley Dickinson. “First Evidence of a Menstruating Rodent: The Spiny Mouse (Acomys Cahirinus).” bioRxiv (June 3, 2016).

Majerus Did Not Need a Randomized Trial to Know that Aspirin Prevents Heart Attacks

(p. A21) Philip W. Majerus, a biochemist who was credited as being the first to theorize that taking small doses of aspirin regularly can prevent heart attacks and strokes in vulnerable patients, died last Wednesday [June 8, 2016] at his home in St. Louis. . . .
. . .
Even before his findings were confirmed in a study by other researchers a decade later, Dr. Majerus was taking aspirin daily.
“I was already convinced that aspirin prevented heart attacks,” he recalled in the journal Advances in Biological Regulation in 2014. “I was unwilling to be randomized into a trial where I might end up with the placebo. I refused to participate.”
Dr. Majerus recommended that “all adults should take an aspirin daily unless they are among the few percent of the population that cannot tolerate the drug.” The cardiovascular benefit of aspirin was fully achieved by 50 to 75 milligrams daily, he said, and “there is no evidence that branded aspirin, which is much more expensive, is in any way superior to the generic version.”
Later studies found that for people in their 50s who are vulnerable to heart disease, taking daily doses of aspirin reduces the risk of heart disease.
. . .
Investigating how aspirin inhibited clotting, Dr. Majerus concluded that the medicine modified an enzyme that leads to the formation of a platelet-made molecule that constricts blood vessels and aggregates platelets. The pills’ effect lasts for the platelets’ life span, typically about two weeks.
“Phil Majerus, more than any other individual, has produced the most original body of work on biochemistry of platelets as it relates to thrombosis,” Prof. Joseph L. Goldstein, a Nobel laureate at the University of Texas Southwestern Medical Center in Dallas, said when the Bristol-Myers Squibb Award was announced.

For the full obituary, see:
SAM ROBERTS. “Dr. Philip Majerus, Who Recognized Heart Benefits of Aspirin, Is Dead at 79.” The New York Times (Weds., JUNE 15, 2016): A23.
(Note: ellipses, and bracketed date, added.)
(Note: the online version of the obituary has the date JUNE 14, 2016, and has the title “Dr. Philip Majerus, Who Discerned Aspirin’s Heart Benefits, Dies at 79.”)

Half of Important Psychology Articles Could Not Be Replicated

(p. A1) The past several years have been bruising ones for the credibility of the social sciences. A star social psychologist was caught fabricating data, leading to more than 50 retracted papers. A top journal published a study supporting the existence of ESP that was widely criticized. The journal Science pulled a political science paper on the effect of gay canvassers on voters’ behavior because of concerns about faked data.
Now, a painstaking yearslong effort to reproduce 100 studies published in three leading psychology journals has found that more than half of the findings did not hold up when retested. The analysis was done by research psychologists, many of whom volunteered their time to double-check what they considered important work. Their conclusions, reported Thursday [August 27, 2015] in the journal Science, have confirmed the worst fears of scientists who have long worried that the field needed a strong correction.
The vetted studies were considered part of the core knowledge by which scientists understand the dynamics of personality, relationships, learning and memory. Therapists and educators rely on such findings to help guide decisions, and the fact that so many of the studies were called into question could sow doubt in the scientific underpinnings of their work.

For the full story, see:
BENEDICT CAREY. “Psychology’s Fears Confirmed: Rechecked Studies Don’t Hold Up.” The New York Times (Fri., AUG. 28, 2015): A1 & A13.
(Note: bracketed date added.)
(Note: the online version of the story has the date AUG. 27, 2015 and has the title “Many Psychology Findings Not as Strong as Claimed, Study Says.”)

The Science article reporting the large number of psychology articles that proved unreplicable, is:
Open Science Collaboration. “Estimating the Reproducibility of Psychological Science.” Science 349, no. 6251 (Aug. 28, 2015): 943.

Many Empirical Research Results Are False

(p. B7) Research on 100 studies in psychology found in 2015 that more than 60% couldn’t be replicated. Similar results have been found in medicine and economics. Campbell Harvey, a professor at Duke University and president of the American Finance Association, estimates that at least half of all “discoveries” in investment research, and financial products based on them, are false.
. . .
Brian Nosek, a psychology professor at the University of Virginia and executive director of the Center for Open Science, a nonprofit seeking to improve research practices, has spent much of the last decade analyzing why so many studies don’t stand up over time.
Because researchers have an incentive to come up with results that are “positive and clean and novel,” he says, they often test a plethora of ideas, throwing out those that don’t appear to work and pursuing those that confirm their own hunches.
If the researchers test enough possibilities, they may find positive results by chance alone — and may fool themselves into believing that luck didn’t determine the outcomes.

For the full commentary, see:
JASON ZWEIG. “Chasing Hot Returns in ‘Smart-Beta’ Can Be Dumb.” The Wall Street Journal (Sat., Feb 13, 2016): B1 & B7.
(Note: ellipsis added.)
(Note: the online version of the commentary has the date Feb 12, 2016, and has the title “Chasing Hot Returns in ‘Smart-Beta’ Funds Can Be a Dumb Idea.”)