Methodology – Page 24 – artdiamondblog.com

Modelers Can Often Obtain the Desired Result

(p. A13) After earning a master’s degree in environmental engineering in 1982, I spent most of the next 10 years building large-scale environmental computer models. My first job was as a consultant to the Environmental Protection Agency. I was hired to build a model to assess the impact of its Construction Grants Program, a nationwide effort in the 1970s and 1980s to upgrade sewer-treatment plants.
The computer model was huge–it analyzed every river, sewer treatment plant and drinking-water intake (the places in rivers where municipalities draw their water) in the country. I’ll spare you the details, but the model showed huge gains from the program as water quality improved dramatically. By the late 1980s, however, any gains from upgrading sewer treatments would be offset by the additional pollution load coming from people who moved from on-site septic tanks to public sewers, which dump the waste into rivers. Basically the model said we had hit the point of diminishing returns.
When I presented the results to the EPA official in charge, he said that I should go back and “sharpen my pencil.” I did. I reviewed assumptions, tweaked coefficients and recalibrated data. But when I reran everything the numbers didn’t change much. At our next meeting he told me to run the numbers again.
After three iterations I finally blurted out, “What number are you looking for?” He didn’t miss a beat: He told me that he needed to show $2 billion of benefits to get the program renewed. I finally turned enough knobs to get the answer he wanted, and everyone was happy.
. . .
There are no exact values for the coefficients in models such as these. There are only ranges of potential values. By moving a bunch of these parameters to one side or the other you can usually get very different results, often (surprise) in line with your initial beliefs.

For the full commentary, see:
ROBERT J. CAPRARA. “OPINION; Confessions of a Computer Modeler; Any model, including those predicting climate doom, can be tweaked to yield a desired result. I should know.” The Wall Street Journal (Weds., July 9, 2014): A13.
(Note: ellipsis added.)
(Note: the online version of the commentary has the date July 8, 2014.)

“Different Structural Models Can Fit Aggregate Macroeconomic Data About Equally Well”

(p. 1149) There is an apparent lack of encompassing-forecasting and economic models that can explain the facts uniformly well across business cycles. This is perhaps an inevitable outcome given the changing nature of business cycles. The fact that business cycles are not all alike naturally means that variables that predict activity have a performance that is episodic. Notably, we find that term spreads were good predictors of economic activity in the 1970s and 1980s, but that credit spreads have fared better more recently. This is of course a challenge for forecasters, as we do (p. 1150) not know the origins of future business cycle fluctuations. Much needs to be learned to determine which and how financial variables are to be monitored in real time especially in an evolving economy when historical data do not provide adequate guidance.
Explanations for the Great Recessions usually involve some form of nonlinearity. The sudden nature of the downturn following the collapse of Lehman is consistent with nonlinearity being part of the transmission mechanism. At the same time, we lack robust evidence of nonlinearity from aggregate low-frequency macroeconomic data. Essentially, there is an identification issue as different structural models can fit aggregate macroeconomic data about equally well.

For the full article, see:
Ng, Serena, and Jonathan H. Wright. “Facts and Challenges from the Great Recession for Forecasting and Macroeconomic Modeling.” Journal of Economic Literature 51, no. 4 (Dec. 2013): 1120-54.

Climate Models Allow “the Modeler to Obtain Almost Any Desired Result”

Integrated assessment models (IAMs) are the commonly-used models that attempt to integrate climate science models with economic effect models. In the passage quoted below, “SCC” stands for “social cost of carbon.”

(p. 870) I have argued that IAMs are of little or no value for evaluating alternative climate change policies and estimating the SCC. On the contrary, an IAM-based analysis suggests a level of knowledge and precision that is nonexistent, and allows the modeler to obtain almost any desired result because key inputs can be chosen arbitrarily.

As I have explained, the physical mechanisms that determine climate sensitivity involve crucial feedback loops, and the parameter values that determine the strength of those feedback loops are largely unknown. When it comes to the impact of climate change, we know even less. IAM damage functions are completely made up, with no theoretical or empirical foundation. They simply reflect common beliefs (which might be wrong) regarding the impact of 2º C or 3º C of warming, and can tell us nothing about what might happen if the temperature increases by 5º C or more. And yet those damage functions are taken seriously when IAMs are used to analyze climate policy. Finally, IAMs tell us nothing about the likelihood and nature of catastrophic outcomes, but it is just such outcomes that matter most for climate change policy. Probably the best we can do at this point is come up with plausible estimates for probabilities and possible impacts of catastrophic outcomes. Doing otherwise is to delude ourselves.

For the full article, see:
Pindyck, Robert S. “Climate Change Policy: What Do the Models Tell Us?” Journal of Economic Literature 51, no. 3 (Sept. 2013): 860-72.

Environmentalists Seek to Silence Those Who Dare to Disagree

(p. A13) Surely, some kind of ending is upon us. Last week climate protesters demanded the silencing of Charles Krauthammer for a Washington Post column that notices uncertainties in the global warming hypothesis. In coming weeks a libel trial gets under way brought by Penn State’s Michael Mann, author of the famed hockey stick, against National Review, the Competitive Enterprise Institute, writer Rand Simberg and roving commentator Mark Steyn for making wisecracks about his climate work. The New York Times runs a cartoon of a climate “denier” being stabbed with an icicle.
These are indications of a political movement turned to defending its self-image as its cause goes down the drain.

For the full commentary, see:
HOLMAN W. JENKINS, JR. “BUSINESS WORLD; Personal Score-Settling Is the New Climate Agenda; The cause of global carbon regulation may be lost, but enemies still can be punished.” The Wall Street Journal (Sat., March 1, 2014): A13.
(Note: the online version of the commentary has the date Feb. 28, 2014, and has the title “BUSINESS WORLD; Jenkins: Personal Score-Settling Is the New Climate Agenda; The cause of global carbon regulation may be lost, but enemies still can be punished.”)

The Krauthammer column that the environmentalists do not want you to read:
Krauthammer, Charles. “The Myth of ‘Settled Science’.” The Washington Post (Fri., Feb. 21, 2014): A19.

Many Important Medical Articles Cannot Be Replicated

The standard scientific method is more fallible, and less logically rigorous, than is generally admitted. One implication is to strengthen the case for allowing patients considerable freedom in choosing their own treatments.

(p. D1) It has been jarring to learn in recent years that a reproducible result may actually be the rarest of birds. Replication, the ability of another lab to reproduce a finding, is the gold standard of science, reassurance that you have discovered something true. But that is getting harder all the time. With the most accessible truths already discovered, what remains are often subtle effects, some so delicate that they can be conjured up only under ideal circumstances, using highly specialized techniques.
Fears that this is resulting in some questionable findings began to emerge in 2005, when Dr. John P. A. Ioannidis, a kind of meta-scientist who researches research, wrote a paper pointedly titled “Why Most Published Research Findings Are False.”
. . .
. . . he published another blockbuster, examining more than a decade’s worth of highly regarded papers — the effect of a daily aspirin on cardiac disease, for example, or the risks of hormone replacement therapy for older women. He found that a large proportion of the conclusions were undermined or contradicted by later studies.
His work was just the beginning. Concern about the problem has reached the point that the journal Nature has assembled an archive, filled with reports and analyses, called Challenges in Irreproducible Research.
Among them is a paper in which C. Glenn Begley, who is chief scientific officer at TetraLogic Pharmaceuticals, described an experience he had while at Amgen, another drug company. He and his colleagues could not replicate 47 of 53 landmark papers about cancer. Some of the results could not be reproduced even with the help of the original scientists working in their own labs.

For the full commentary, see:
GEORGE JOHNSON. “Raw Data; New Truths That Only One Can See.” The New York Times (Tues., JAN. 21, 2014): D1 & D6.
(Note: ellipses added.)
(Note: the online version of the commentary has the date JAN. 20, 2014.)

The first Ioannidis article mentioned above is:
Ioannidis, John P. A. “Why Most Published Research Findings Are False.” PLoS Medicine 2, no. 8 (August 2005): 696-701.

The second Ioannidis article mentioned above is:
Ioannidis, John P. A. “Contradicted and Initially Stronger Effects in Highly Cited Clinical Research.” JAMA 294, no. 2 (July 13, 2005): 218-28.

The Begley article mentioned above is:
Begley, C. Glenn, and Lee M. Ellis. “Drug Development: Raise Standards for Preclinical Cancer Research.” Nature 483, no. 7391 (March 29, 2012): 531-33.

Better to Fail at Solving a Big Problem, than to Succeed at a Minor One?

Source of book image: http://ecx.images-amazon.com/images/I/61s10qMqpxL._SL1400_.jpg

Francis Collins, head of the NIH, discusses a favorite book of 2013:

(p. C6) Taking risks is part of genius, and genius is not immune to bloopers. Mario Livio’s “Brilliant Blunders” leads us through the circumstances that surrounded famous gaffes. . . . Mr. Livio helps us see that such spectacular errors are opportunities rather than setbacks. There’s a lesson for young scientists here. Boldly attacking problems of fundamental significance can have more impact than pursuing precise solutions to minor questions–even if there are a few bungles along the way.

For the full article, see:
“12 Months of Reading; We asked 50 of our friends–from April Bloomfield to Mike Tyson–to name their favorite books of 2013.” The Wall Street Journal (Sat., Dec. 14, 2013): C6 & C9-C12.
(Note: the online version of the article has the date Dec. 13, 2013.)

The book that Collins praises is:
Livio, Mario. Brilliant Blunders: From Darwin to Einstein – Colossal Mistakes by Great Scientists That Changed Our Understanding of Life and the Universe. New York: Simon & Schuster, 2013.

Would Science Progress Faster If It Were Less Academic and More Entrepreneurial?

Source of caption and photo: online version of the NYT article quoted and cited below.

(p. D5) There is Big Science, defined as science that gets the big bucks. There is tried and true science, which, from an adventurous dissident’s point of view, is boldly going where others have gone before but extending the prevailing knowledge by a couple of decimal places (a safe approach for dissertation writers and grant seekers).

Then there is bootstrap science, personified by Gene Shinn, who retired in 2006 after 31 years with the United States Geological Survey and 15 years with a research arm of the Shell Oil Company.
. . .
Without a Ph.D. and often without much financing, Mr. Shinn published more than 120 peer-reviewed papers that helped change many experts’ views on subjects like how coral reefs expand and the underwater formation of limestone. Some of his papers, at odds with established scientific views, were initially rejected, only to be seen later as visionary.
His bootstrap ingredients included boundless curiosity, big ideas — “gee-whiz science,” he calls it — persistence, a sure hand at underwater demolition (dynamite was comparatively easy to come by in those remarkably innocent days) and versatility at improvising core-sampling equipment on tight budgets. The ability to enlist the talents of other scientists, many with doctorates, who shared his love of hands-on field work and his impatience with official rules and permits added to the mix.

For the full review, see:
MICHAEL POLLAK. “BOOKS; Science on His Own Terms.” The New York Times (Tues., November 5, 2013): D5.
(Note: the online version of the review has the date November 4, 2013.)

Book under review:
Shinn, Eugene A. Bootstrap Geologist: My Life in Science. Gainesville, FL: University Press of Florida, 2013.

The Use of Note Cards to Structure Writing

(p. A21) I tell college students that by the time they sit down at the keyboard to write their essays, they should be at least 80 percent done. That’s because “writing” is mostly gathering and structuring ideas.
For what it’s worth, I structure geographically. I organize my notes into different piles on the rug in my living room. Each pile represents a different paragraph in my column. The piles can stretch on for 10 feet to 16 feet, even for a mere 806-word newspaper piece. When “writing,” I just pick up a pile, synthesize the notes into a paragraph, set them aside and move on to the next pile. If the piece isn’t working, I don’t try to repair; I start from scratch with the same topic but an entirely new structure.
The longtime New Yorker writer John McPhee wonderfully described his process in an essay just called “Structure.” For one long article, McPhee organized his notecards on a 32-square-foot piece of plywood. He also describes the common tension between chronology and theme (my advice: go with chronology). His structures are brilliant, but they far too complex for most of us. The key thing is he lets you see how a really fine writer thinks about the core problem of writing, which takes place before the actual writing.

For the full commentary, see:
DAVID BROOKS. “The Sidney Awards, Part 2.” The New York Times (Tues., December 31, 2013): A21. [National Edition]
(Note: the online version of the commentary has the date December 30, 2013.)

The article praised by Brooks is:
McPhee, John. “Structure.” The New Yorker (Jan. 14, 2013): 46-55.

Key to Google: “Both Larry and Sergey Were Montessori Kids”

(p. 121) [Marissa Mayer] conceded that to an outsider, Google’s new-business process might indeed look strange. Google spun out projects like buckshot, blasting a spray and using tools and measurements to see what it hit. And sometimes it did try ideas that seemed ill suited or just plain odd. Finally she burst out with her version of the corporate Rosebud. “You can’t understand Google,” she said, “unless you know that both Larry and Sergey were Montessori kids.”
“Montessori” refers to schools based on the educational philosophy of Maria Montessori, an Italian physician born in 1870 who believed that children should be allowed the freedom to pursue what interested them.
(p. 122) “It’s really ingrained in their personalities,” she said. “To ask their own questions, do their own things. To disrespect authority. Do something because it makes sense, not because some authority figure told you. In Montessori school you go paint because you have something to express or you just want to do it that afternoon, not because the teacher said so. This is really baked into how Larry and Sergey approach problems. They’re always asking ‘Why should it be like that?’ It’s the way their brains were programmed early on.”

Source:
Levy, Steven. In the Plex: How Google Thinks, Works, and Shapes Our Lives. New York: Simon & Schuster, 2011.
(Note: bracketed name added.)

Ignoring Einstein’s Mistakes by Deifying Him, Makes Us Forget His Struggles

Source of book image: http://ecx.images-amazon.com/images/I/41zyL4LVYxL.jpg

(p. A13) Mr. Ohanian finds that four out of five of the seminal papers that Einstein produced in the so-called “miracle year” of 1905, when he was working as a patent inspector in Zurich, were “infested with flaws.”
. . .
. . . he notes Einstein’s errors for a purpose, showing us why his achievement was all the greater for them.
In this Mr. Ohanian provides a useful corrective, for there is a tendency, even today, to deify Einstein and other men of genius, treating them as if they were immortal gods. Einstein himself objected to the practice even as he reveled in his fame. “It is not fair,” he once observed, “to select a few individuals for boundless admiration and to attribute superhuman powers of mind and of character to them.” In doing so, ironically, we make less of the person, not more, forgetting and simplifying their struggle.
. . .
. . . Einstein’s ability to make use of his mistakes as “stepping stones and shortcuts” was central to his success, in Mr. Ohanian’s view. To see Einstein’s wanderings not as the strides of a god-like genius but as the steps and missteps of a man — fallible and imperfect — does not diminish our respect for him but rather enhances it.

For the full review, see:
McMahon, Darrin M. “BOOKSHELF; Great and Imperfect.” The Wall Street Journal (Fri., September 5, 2008): A13.
(Note: ellipses added.)

The book under review is:
Ohanian, Hans C. Einstein’s Mistakes: The Human Failings of Genius. New York: W. W. Norton & Company, 2008.

We Should Disenthrall Ourselves of False Scientific Certainties

Source of book image: http://2.bp.blogspot.com/-ELpfH2bTO7c/Tb53WpKuDxI/AAAAAAAADrE/Zq8BQiiasJc/s640/An+Optimists+Tour+of+the+Future+Cover.jpg

(p. C4) Among the scientific certainties I have had to unlearn: that upbringing strongly shapes your personality; that nurture is the opposite of nature; that dietary fat causes obesity more than dietary carbohydrate; that carbon dioxide has been the main driver of climate change in the past.

I came across a rather good word for this kind of unlearning–“disenthrall”–in Mark Stevenson’s book “An Optimist’s Tour of the Future,” published just this week. Mr. Stevenson borrows it from Abraham Lincoln, whose 1862 message to Congress speaks of disenthralling ourselves of “the dogmas of the quiet past” in order to “think anew.”
Mr. Stevenson’s disenthrallment comes in the course of a series of sharp and fascinating interviews with technological innovators and scientific visionaries. This disenthralls him of the pessimism about the future and nostalgia about the past that he barely realized he had and whose “fingers reach deep into [his] soul.” It eventually turns him into an optimist almost as ludicrously sanguine about the 21st century as I am: “I steadfastly refuse to believe that human society can’t grow, improve and learn; that it can’t embrace change and remake the world better.”
Along the way, Mr. Stevenson is struck by other examples of how the way he thinks and reasons is “in thrall to a world that is passing.” The first of these bad habits is linear thinking about the future. . . .
We expect to see changes coming gradually, but because things like computing power or the cheapness of genome sequencing change exponentially, technologies can go from impossible to cheap quite suddenly and with little warning.

For the full commentary, see:
MATT RIDLEY. “MIND & MATTER; A Key Lesson of Adulthood: The Need to Unlearn.” The Wall Street Journal (Sat., February 5, 2011): C4.
(Note: ellipsis added.)

The book praised by Ridley, in the passages quoted above, is:
Stevenson, Mark. An Optimist’s Tour of the Future: One Curious Man Sets out to Answer “What’s Next?”. New York: Avery, 2011.