Category: Methodology

“Science Is Not a Body of Infallible Work, of Immutable Laws”

(p. 1) . . . , “Failure: Why Science Is So Successful” is a breath of contemplative fresh air. Stuart ­Fire­stein, a professor in the department of biological sciences at Columbia University, is best known for his work on ignorance, including inviting scientists to speak to his students about what they don’t know. In a tone reminiscent of Lewis Thomas’s “The Lives of a Cell,” the book is a collection of loosely interwoven meditations on failure and scientific method.
. . .
If we succeed by failing, then we should be freed from the monolithic road to academic tenure; science should be taught as an adventure in failure. With a delightful combination of feigned naïveté and keen eye for the messy ways that great discoveries occur, he goes so far as to suggest writing a grant proposal in which you promise to fail better. He knows this isn’t how the world works, but nevertheless argues that change will take place “when we cease, or at least reduce, our devotion to facts and collections of them, when we decide that science education is not a memorization marathon, when we — scientists and nonscientists — recognize that science is not a body of infallible work, of immutable laws of facts. . . . And that most of what there is to know is still unknown.”

For the full review, see:
ROBERT A. BURTON. “Error Messages.”The New York Times, SundayReview Section (Sun., Jan. 3, 2016): 8.
(Note: first two ellipses added; third ellipsis in original.)
(Note: the online version of the review has the date DEC. 29, 2015, and has the title “‘Black Box Thinking’ and ‘Failure: Why Science Is So Successful’.”)

The book under review, is:
Firestein, Stuart. Failure: Why Science Is So Successful. New York: Oxford University Press, 2016.

Serendipity May Be Source of 50% of Patents

(p. 1) A surprising number of the conveniences of modern life were invented when someone stumbled upon a discovery or capitalized on an accident: the microwave oven, safety glass, smoke detectors, artificial sweeteners, (p. 4) X-ray imaging. Many blockbuster drugs of the 20th century emerged because a lab worker picked up on the “wrong” information.
. . .
(p. 5) So how many big ideas emerge from spills, crashes, failed experiments and blind stabs? One survey of patent holders (the PatVal study of European inventors, published in 2005) found that an incredible 50 percent of patents resulted from what could be described as a serendipitous process. Thousands of survey respondents reported that their idea evolved when they were working on an unrelated project — and often when they weren’t even trying to invent anything. This is why we need to know far more about the habits that transform a mistake into a breakthrough.
. . .
A number of pioneering scholars have already begun this work, but they seem to be doing so in their own silos and without much cross-talk. In a 2005 paper (“Serendipitous Insights Involving Nonhuman Primates”), two experts from the Washington National Primate Research Center in Seattle cataloged the chance encounters that yielded new insights from creatures like the pigtail macaque. Meanwhile, the authors of a paper titled “On the Exploitation of Serendipity in Drug Discovery” puzzled over the reasons the 1950s and ’60s saw a bonanza of breakthroughs in psychiatric medication, and why that run of serendipity ended.

For the full commentary, see:
PAGAN KENNEDY. “How to Cultivate the Art of Serendipity.” The New York Times, SundayReview Section (Sun., JAN. 3, 2016): 1 & 4-5.
(Note: ellipses added.)
(Note: the online version of the commentary has the date JAN. 2, 2016, and has the title “Cultivating the Art of Serendipity.”)

Pagan’s commentary is based on her book:
Kennedy, Pagan. Inventology: How We Dream up Things That Change the World. New York: Houghton Mifflin Harcourt Publishing Co., 2016.

Scientific Insight Requires Hard Work More than Easy Epiphany

(p. A21) The myth of the finches obscures the qualities that were really responsible for Darwin’s success: the grit to formulate his theory and gather evidence for it; the creativity to seek signs of evolution in existing animals, rather than, as others did, in the fossil record; and the open-mindedness to drop his belief in creationism when the evidence against it piled up.
The mythical stories we tell about our heroes are always more romantic and often more palatable than the truth. But in science, at least, they are destructive, in that they promote false conceptions of the evolution of scientific thought.
Of the tale of Newton and the apple, the historian Richard S. Westfall wrote, “The story vulgarizes universal gravitation by treating it as a bright idea … A bright idea cannot shape a scientific tradition.” Science is just not that simple and it is not that easy.
. . .
Even if we are not scientists, every day we are challenged to make judgments and decisions about technical matters like vaccinations, financial investments, diet supplements and, of course, global warming. If our discourse on such topics is to be intelligent and productive, we need to dip below the surface and grapple with the complex underlying issues. The myths can seduce one into believing there is an easier path, one that doesn’t require such hard work.
But even beyond issues of science, there is a broader lesson to learn, . . . . We all run into difficult problems in life, and we will be happier and more successful if we appreciate that the answers often aren’t quick, or easy.

For the full commentary, see:
LEONARD MLODINOW. “It Is, in Fact, Rocket Science.” The Wall Street Journal (Sat., MAY 16, 2015): A21.
(Note: ellipsis internal to third quoted paragraph, in original; other ellipses, added.)
(Note: the online version of the commentary was updated on MAY 15, 2015.)

Mlodinow’s book, related to the commentary quoted above, is:
Mlodinow, Leonard. The Upright Thinkers: The Human Journey from Living in Trees to Understanding the Cosmos. New York: Pantheon Books, 2015.

Lives Lost Due to Peer Review Delays

(p. A25) In this age of instant information, medicine remains anchored in the practice of releasing new knowledge at a deliberate pace. It’s time for medical scientists to think differently about how quickly they alert the public to breakthrough findings.
Last week the National Institutes of Health announced that it had prematurely ended a large national study of how best to treat people with high blood pressure because of its exceptional results.
In this trial of more than 9,000 people age 50 and older with high blood pressure, an aggressive treatment strategy to keep systolic blood pressure below 120 was compared with a conventional one aimed at keeping it below 140. The subjects all had a high risk of heart attacks, stroke and heart failure. The N.I.H. concluded, six years into a planned eight-year study, that for these patients, pushing blood pressure down far below currently recommended levels was very beneficial.
. . .

The new information may justify a more vigorous strategy for treating blood pressure, but for now doctors and patients have been left with incomplete results, some headlines and considerable uncertainty about whether to modify current treatments.
Medicine needs to change its approach to releasing new, important information. Throughout science we are seeing more rapid modes of communication. The traditional approach was not to publish until everything was finalized and ready to be chiseled in stone. But these sorts of delays are unnecessary with the Internet. Moreover, although all the trial data has yet to be tabulated, an analysis was considered sufficiently definitive to lead independent experts to stop the multimillion-dollar study.
We believe that when there is such strong evidence for a major public health condition, there should be rapid release of the information that led to the decision to stop the trial. This approach could easily be accomplished by placing the data on the N.I.H. website or publishing the data on such platforms as bioRxiv.org, which enables fast, open review by the medical community.
. . .
Kudos to the scientists who conducted such a large, complex and important study with what will be likely to have lifesaving consequences for a condition that can be treated easily in most patients. Now the medical community needs to adopt a new approach in situations like this one to disseminate lifesaving results in a timely, comprehensive and transparent way. Lives depend on it.

For the full commentary, see:
ERIC J. TOPOL and HARLAN M. KRUMHOLZ. “Don’t Sit on Medical Breakthroughs.” The New York Times (Fri., SEPT. 17, 2015): A25.
(Note: ellipses added.)
(Note: the online version of the commentary has the date SEPT. 17, 2015, and the title “Don’t Delay News of Medical Breakthroughs.”)

Those Who Use “Consensus” Argument on Global Warming, Should Endorse Genetically Modified Food

(p. B3) NAIROBI, Kenya — Mohammed Rahman doesn’t know it yet, but his small farm in central Bangladesh is globally significant. Mr. Rahman, a smallholder farmer in Krishnapur, about 60 miles northwest of the capital, Dhaka, grows eggplant on his meager acre of waterlogged land.
As we squatted in the muddy field, examining the lush green foliage and shiny purple fruits, he explained how, for the first time this season, he had been able to stop using pesticides. This was thanks to a new pest-resistant variety of eggplant supplied by the government-run Bangladesh Agricultural Research Institute.
Despite a recent hailstorm, the weather had been kind, and the new crop flourished. Productivity nearly doubled. Mr. Rahman had already harvested the small plot 10 times, he said, and sold the brinjal (eggplant’s name in the region) labeled “insecticide free” at a small premium in the local market. Now, with increased profits, he looked forward to being able to lift his family further out of poverty. I could see why this was so urgent: Half a dozen shirtless kids gathered around, clamoring for attention. They all looked stunted by malnutrition.
. . .
I, . . . , was once in [the] . . . activist camp. A lifelong environmentalist, I opposed genetically modified foods in the past. Fifteen years ago, I even participated in vandalizing field trials in Britain. Then I changed my mind.
After writing two books on the science of climate change, I decided I could no longer continue taking a pro-science position on global warming and an anti-science position on G.M.O.s.
There is an equivalent level of scientific consensus on both issues, I realized, that climate change is real and genetically modified foods are safe. I could not defend the expert consensus on one issue while opposing it on the other.

For the full commentary, see:
MARK LYNAS. “How I Got Converted to G.M.O. Food.” The New York Times, SundayReview Section (Sun., APRIL 26, 2015): 5.
(Note: ellipses, and bracketed word, added.)
(Note: the online version of the commentary has the date APRIL 24, 2015.)

“Strong-Willed Scientists Overstated the Significance of Their Studies”

The New York Times seems open to the idea that strong-willed scientists might overstate their results in science food studies. I wonder if The New York Times would be open to the same possibility in science climate studies?

(p. A19) For two generations, Americans ate fewer eggs and other animal products because policy makers told them that fat and cholesterol were bad for their health. Now both dogmas have been debunked in quick succession.
. . .
Epidemiological data can be used to suggest hypotheses but not to prove them.
Instead of accepting that this evidence was inadequate to give sound advice, strong-willed scientists overstated the significance of their studies.
Much of the epidemiological data underpinning the government’s dietary advice comes from studies run by Harvard’s school of public health. In 2011, directors of the National Institute of Statistical Sciences analyzed many of Harvard’s most important findings and found that they could not be reproduced in clinical trials.
It’s no surprise that longstanding nutritional guidelines are now being challenged.
In 2013, government advice to reduce salt intake (which remains in the current report) was contradicted by an authoritative Institute of Medicine study. And several recent meta-analyses have cast serious doubt on whether saturated fats are linked to heart disease, as the dietary guidelines continue to assert.
Uncertain science should no longer guide our nutrition policy. Indeed, cutting fat and cholesterol, as Americans have conscientiously done, may have even worsened our health.

For the full commentary, see:
NINA TEICHOLZ. “The Government’s Bad Diet Advice.” The New York Times (Sat., FEB. 21, 2015): A19.
(Note: ellipsis added.)
(Note: the online version of the commentary has the date FEB. 20, 2015.)

Experts Are Paid “to Sound Cocksure” Even When They Do Not Know

(p. B1) I think Philip Tetlock’s “Superforecasting: The Art and Science of Prediction,” co-written with the journalist Dan Gardner, is the most important book on decision making since Daniel Kahneman’s “Thinking, Fast and Slow.” (I helped write and edit the Kahneman book but receive no royalties from it.) Prof. Kahneman agrees. “It’s a manual to systematic thinking in the real world,” he told me. “This book shows that under the right conditions regular people are capable of improving their judgment enough to beat the professionals at their own game.”
The book is so powerful because Prof. Tetlock, a psychologist and professor of management at the University of Pennsylvania’s Wharton School, has a remarkable trove of data. He has just concluded the first stage of what he calls the Good Judgment Project, which pitted some 20,000 amateur forecasters against some of the most knowledgeable experts in the world.
The amateurs won–hands down.
. . .
(p. B7) The most careful, curious, open-minded, persistent and self-critical–as measured by a battery of psychological tests–did the best.
. . .
Most experts–like most people–“are too quick to make up their minds and too slow to change them,” he says. And experts are paid not just to be right, but to sound right: cocksure even when the evidence is sparse or ambiguous.

For the full review, see:
JASON ZWEIG. “The Trick to Making Better Forecasts.” The Wall Street Journal (Sat., Sept. 26, 2015): B1 & B7.
(Note: ellipses added.)
(Note: the online version of the review has the date Sept. 25, 2015.)

The book under review, is:
Tetlock, Philip E., and Dan Gardner. Superforecasting: The Art and Science of Prediction. New York: Crown, 2015.

“Stunned” Geophysicists Are Headed “Back to the Drawing Board”

(p. A3) Bringing the blur of a distant world into sharp focus, NASA unveiled its first intimate images of Pluto on Wednesday [July 15, 2015], revealing with startling clarity an eerie realm where frozen water rises in mountains up to 11,000 feet high.
. . .
At a briefing held Wednesday [July 15, 2015] at the Johns Hopkins Applied Physics Laboratory in Laurel, Md., mission scientists said they were stunned by what the images reveal.
“It is going to send a lot of geophysicists back to the drawing board,” said Alan Stern, the New Horizons project’s principal investigator, from the Southwest Research Institute in Boulder, Colo.

For the full story, see:
ROBERT LEE HOTZ. “Across 3 Billion Miles of Space, NASA Probe Sends Close-Ups of Pluto’s Icy Mountains.” The Wall Street Journal (Thurs., JULY 16, 2015): A3.
(Note: ellipsis, and bracketed dates, added.)
(Note: the online version of the article has the date JULY 15, 2015, has the title “NASA Releases Close-Up Pictures of Pluto and Its Largest Moon, Charon,” and has some different wording than the print version. The quote above follows the online version.)

Rather than Debate Global Warming Skeptics, Some Label them “Denialists” to “Link Them to Holocaust Denial”

(p. D2) The contrarian scientists like to present these upbeat scenarios as the only plausible outcomes from runaway emissions growth. Mainstream scientists see them as being the low end of a range of possible outcomes that includes an alarming high end, and they say the only way to reduce the risks is to reduce emissions.
The dissenting scientists have been called “lukewarmers” by some, for their view that Earth will warm only a little. That is a term Dr. Michaels embraces. “I think it’s wonderful!” he said. He is working on a book, “The Lukewarmers’ Manifesto.”
When they publish in scientific journals, presenting data and arguments to support their views, these contrarians are practicing science, and perhaps the “skeptic” label is applicable. But not all of them are eager to embrace it.
“As far as I can tell, skepticism involves doubts about a plausible proposition,” another of these scientists, Richard S. Lindzen, told an audience a few years ago. “I think current global warming alarm does not represent a plausible proposition.”
. . .
It is perhaps no surprise that many environmentalists have started to call them deniers.
The scientific dissenters object to that word, claiming it is a deliberate attempt to link them to Holocaust denial. Some academics sharply dispute having any such intention, but others have started using the slightly softer word “denialist” to make the same point without stirring complaints about evoking the Holocaust.

For the full commentary, see:
Justin Gillis. “BY DEGREES; Verbal Warming: Labels in the Climate Debate.” The New York Times (Tues., FEB. 17, 2015): D1-D2.
(Note: ellipsis added.)
(Note: the online version of the commentary has the date FEB. 12 (sic), 2015.)

“Big Data” Does Not Tell Us What to Measure, and Ignores What Cannot Be Measured

(p. 6) BIG data will save the world. How often have we heard that over the past couple of years? We’re pretty sure both of us have said something similar dozens of times in the past few months.
If you’re trying to build a self-driving car or detect whether a picture has a cat in it, big data is amazing. But here’s a secret: If you’re trying to make important decisions about your health, wealth or happiness, big data is not enough.
The problem is this: The things we can measure are never exactly what we care about. Just trying to get a single, easy-to-measure number higher and higher (or lower and lower) doesn’t actually help us make the right choice. For this reason, the key question isn’t “What did I measure?” but “What did I miss?”
. . .
So what can big data do to help us make big decisions? One of us, Alex, is a data scientist at Facebook. The other, Seth, is a former data scientist at Google. There is a special sauce necessary to making big data work: surveys and the judgment of humans — two seemingly old-fashioned approaches that we will call small data.
Facebook has tons of data on how people use its site. It’s easy to see whether a particular news feed story was liked, clicked, commented on or shared. But not one of these is a perfect proxy for more important questions: What was the experience like? Did the story connect you with your friends? Did it inform you about the world? Did it make you laugh?
(p. 7) To get to these measures, Facebook has to take an old-fashioned approach: asking. Every day, hundreds of individuals load their news feed and answer questions about the stories they see there. Big data (likes, clicks, comments) is supplemented by small data (“Do you want to see this post in your News Feed?”) and contextualized (“Why?”).
Big data in the form of behaviors and small data in the form of surveys complement each other and produce insights rather than simple metrics.
. . .
Because of this need for small data, Facebook’s data teams look different than you would guess. Facebook employs social psychologists, anthropologists and sociologists precisely to find what simple measures miss.
And it’s not just Silicon Valley firms that employ the power of small data. Baseball is often used as the quintessential story of data geeks, crunching huge data sets, replacing fallible human experts, like scouts. This story was made famous in both the book and the movie “Moneyball.”
But the true story is not that simple. For one thing, many teams ended up going overboard on data. It was easy to measure offense and pitching, so some organizations ended up underestimating the importance of defense, which is harder to measure. In fact, in his book “The Signal and the Noise,” Nate Silver of fivethirtyeight.com estimates that the Oakland A’s were giving up 8 to 10 wins per year in the mid-1990s because of their lousy defense.
. . .
Human experts can also help data analysts figure out what to look for. For decades, scouts have judged catchers based on their ability to frame pitches — to make the pitch appear more like a strike to a watching umpire. Thanks to improved data on pitch location, analysts have recently checked this hypothesis and confirmed that catchers differ significantly in this skill.

For the full commentary, see:
ALEX PEYSAKHOVICH and SETH STEPHENS-DAVIDOWITZ. “How Not to Drown in Numbers.” The New York Times, SundayReview Section (Sun., MAY 3, 2015): 6-7.
(Note: ellipses added.)
(Note: the online version of the commentary has the date MAY 2, 2015.)

Insights More Likely When Mood Is Positive and Distractions Few

If insights are more likely in the absence of distractions, then why are business executives so universally gung-ho on imposing on their workers the open office space layouts that are guaranteed to maximize distractions?

(p. C7) We can’t put a mathematician inside an fMRI machine and demand that she have a breakthrough over the course of 20 minutes or even an hour. These kinds of breakthroughs are too mercurial and rare to be subjected to experimentation.

We are, however, able to study the phenomenon more generally. Enter John Kounios and Mark Beeman, two cognitive neuroscientists and the authors of the “The Eureka Factor.” Messrs. Kounios and Beeman focus their book on the science behind insights and how to cultivate them.
As Mr. Irvine recognizes, studying insights in the lab is difficult. But it’s not impossible. Scientists have devised experiments that can provoke in subjects these kinds of insights, ones that feel genuine but occur on a much smaller scale.
. . .
The book includes some practical takeaways of how to improve our odds of getting insights as well. Blocking out distractions can create an environment conducive to insights. So can having a positive mood. While many of the suggestions contain caveats, as befits the delicate nature of creativity, ultimately it seems that there are ways to be more open to these moments of insight.

For the full review, see:
SAMUEL ARBESMAN. “Every Man an Archimedes; Insights can seem to appear spontaneously, but fully formed. No wonder the ancients spoke of muses.” The Wall Street Journal (Sat., May 23, 2015): C7.
(Note: ellipsis added.)
(Note: the online version of the review has the date May 22, 2015.)

The book under review, is:
Kounios, John, and Mark Beeman. The Eureka Factor: Aha Moments, Creative Insight, and the Brain. New York: Random House, 2015.