In the Actual Messy Real-World, RCTs Can Be a Fool’s Gold Standard

George Stigler used to emphasize that when we compare markets to the government we often make an unfair comparison. We compare the ideal theoretical government, with the actual messy real-world market. We do the same thing in comparing methods of reaching medical truth. In this analogy the role of the government is played by double-blind randomized clinical trials (RCTs). In ideal theory, RCTs are the “gold standard.” The story quoted from below, shows that a lot can go wrong when RCTs are conducted in the actual messy real-world.

Note that the flaws in the RCT were NOT caught in the peer-review process, but were caught by “online sleuths and physicians,” practitioners with a sense of urgency and mission.

(p. D2) Early this year, a medical journal article caught the attention of cancer patients and doctors worldwide because of its extraordinary conclusion. Simply changing the time of day that immunotherapy was administered appeared to produce a stunning benefit for lung cancer patients.

Those who received IV infusions in the morning had their cancer kept at bay for twice as long as those who got it in the afternoon, according to the results from a clinical trial in China and published in the journal Nature Medicine in February [2026]. The study also reported that the patients lived nearly twice as long.

. . .

But on Wednesday [June 24, 2026], Nature Medicine retracted the study, citing a list of inconsistencies and irregularities in the trial’s design and results.

. . .

Among the issues that the journal cited in its retraction notice: Records that were supposed to be locked before the study started were changed midway through. There were discrepancies between the Chinese version of the study’s plan and the translated version. Every patient remained treated and tracked in the study’s first year, and no one dropped out because of side effects — highly unusual in an oncology study. And unusual patterns were found in the timing of follow-up scans.

“Due to the amount and nature of the problems identified, the editors no longer have confidence in the integrity of the results,” the journal said.

. . .

China’s critics often question the reliability of its biomedical research. Experts said that, similar to studies in the United States, China’s research output spans a wide range in quality: Some Chinese scientists run their studies at the most meticulous standards. Others were said to cut corners.

. . .

The study had enrolled 210 patients with advanced lung cancer at Hunan Cancer Hospital in Changsha, a city in south-central China. Patients were randomly assigned to receive infusions of an immunotherapy — Merck’s blockbuster drug Keytruda or Tyvyt, which is not approved in the United States — either before or after 3 p.m.

The study reported that tumors did not progress for 11 months in patients who received the earlier infusions, compared with six months for those given the later infusions. Patients receiving the earlier infusions lived for 28 months, compared with 17 months for those infused later in the day.

. . .

Dr. Anil Makam, an epidemiologist and health services researcher at the University of California, San Francisco, said such a drastic benefit, if real, would have prompted infusion clinics to overhaul staffing and scheduling to shift appointments to earlier in the day.

“If we believed the effects, it would be malpractice not to,” he said.

But within days after the study’s publication in Nature Medicine, online sleuths and physicians like Dr. Makam began raising concerns on social media and in blog posts. Less than three weeks after the study was published, the journal published an editor’s note saying it was investigating the issues.

In a statement on Thursday [June 25, 2026], Dr. João Monteiro, chief editor of Nature Medicine, which is published by Springer Nature, said, “We are grateful to the research community for bringing these concerns to our attention.”

For the full article, see:

Rebecca Robbins. “‘Too Good to Be True’: Chinese Study on Cancer Therapy Is Retracted.” The New York Times (Tues., June 30, 2026): B3.

(Note: ellipses, and bracketed dates, added.)

(Note: the online version of the Houreld article was updated June 25, 2026, and has the title “‘Too Good to Be True’: A Chinese Study on Timing Cancer Therapy Is Retracted.” Where there is a slight difference in wording, the quotations above follow the print version.)

The Chicago School of Economics Was Once Uniquely Focused on Real World Problems

The Chicago School of Economics, most associated with Milton Friedman and George Stigler, saw itself as different from all the other top graduate programs in economics. At Chicago, the priority was solving applied problems, and only as much mathematics and theory should be used as was necessary to solve them. The other schools prioritized mathematical puzzle-solving and mathematical rigor and sophistication.

For those who might suspect Chicago was full of itself, the non-Chicago economists Arjo Klamar and David Colander dispelled the suspicion in their The Making of an Economist. After thorough interviewing and surveying of graduate students at the five or six top graduate programs, they concluded that graduate students at all but Chicago were cynically discouraged to realize that they were being trained to solve mathematical puzzles, while only those at Chicago still felt that they were being trained to matter in the real world.

I noticed that a recent obituary for the economist Stanley Fischer quotes Fischer as stating some diplomatic confirmation of the Klamar and Colander conclusion:

After earning his Ph.D. at M.I.T. in 1969, Mr. Fischer moved to the University of Chicago as a postdoctoral researcher and assistant professor. “At M.I.T. you did the mathematical work,” he told The New York Times in 1998, “and at Chicago you asked the question of how this applies to the real world” (Hagerty 2025, p. A17).

Alas, I fear that what was once true, is true no longer. I fear that if Klamar and Colander were to repeat their study today, they would find that Chicago has joined the other top programs in prioritizing mathematical puzzle-solving and mathematical rigor and sophistication.

The obituary of Stanley Fischer, quoted above, is:

James R. Hagerty. “Stanley Fischer, 81, Economist Who Helped Defuse Crises, Dies.” The New York Times (Mon., June 2, 2025): A17.

(Note: the online version of the Steve Lohr article was updated June 10, 2025, and has the title “Stanley Fischer, Who Helped Defuse Financial Crises, Dies at 81.”)

The Klamar and Colander book mentioned above is:

Klamer, Arjo, and David Colander. The Making of an Economist. Boulder, CO: Westview Press, 1990.

Time Constraints for Tenure, Promotion, and Funding Decisions Lead Academic Biologists to Over-Study Already-Studied Genes

George Stigler argued that when most economists were self-funded business practitioners economics was more applied and empirical, while after most economists were academics funded by endowments or the government economics became less applied and more formal. [In a quick search I failed to identify the article where Stigler says this–sorry.] A similar point was made to science more broadly by Terence Kealey in his thought-provoking The Economic Laws of Scientific Research. The article quoted below argues persuasively that research on human genes is aligned with the career survival goals of academics, rather than with either the faster advance of science or the quicker cure of diseases like cancer. The alignment could be improved if more of research funding came from a variety of private sources.

(p. D3) In a study published Tuesday [Sept. 18, 2018] in PLOS Biology, researchers at Northwestern University reported that of our 20,000 protein-coding genes, about 5,400 have never been the subject of a single dedicated paper.

Most of our other genes have been almost as badly neglected, the subjects of minor investigation at best. A tiny fraction — 2,000 of them — have hogged most of the attention, the focus of 90 percent of the scientific studies published in recent years.

A number of factors are largely responsible for this wild imbalance, and they say a lot about how scientists approach science.

. . .

It was possible, . . ., that scientists were rationally focusing attention only on the genes that matter most. Perhaps they only studied the genes involved in cancer and other diseases.

That was not the case, it turned out. “There are lots of genes that are important for cancer, but only a small subset of them are being studied,” said Dr. Amaral.

. . .

A long history helps, . . . . The genes that are intensively studied now tend to be the ones that were discovered long ago.

Some 16 percent of all human genes were identified by 1991. Those genes were the subjects of about half of all genetic research published in 2015.

One reason is that the longer scientists study a gene, the easier it gets, noted Thomas Stoeger, a post-doctoral researcher at Northwestern and a co-author of the new report.

“People who study these genes have a head start over scientists who have to make tools to study other genes,” he said.

That head start may make all the difference in the scramble to publish research and land a job. Graduate students who investigated the least studied genes were much less likely to become a principal investigators later in their careers, the new study found.

“All the rewards are set up for you to study what has been well-studied,” Dr. Amaral said.

“With the Human Genome Project, we thought everything was going to change,” he added. “And what our analysis shows is pretty much nothing changed.”

If these trends continue as they have for decades, the human genome will remain a terra incognito for a long time. At this rate, it would take a century or longer for scientists to publish at least one paper on every one of our 20,000 genes.

That slow pace of discovery may well stymie advances in medicine, Dr. Amaral said. “We keep looking at the same genes as targets for our drugs. We are ignoring the vast majority of the genome,” he said.

Scientists won’t change their ways without a major shift in how science gets done, he added. “I can’t believe the system can move in that direction by itself,” he said.

Dr. Stoeger argued that the scientific community should recognize that a researcher who studies the least known genes may need extra time to get results.

“People who do something new need some protection,” he said.

For the full commentary see:

Carl Zimmer. “Matter; The Problem With DNA Research.” The New York Times (Tuesday, September 25, 2018 [sic]): D3.

(Note: ellipses, and bracketed date, added.)

(Note: the online version of the commentary has the date Sept. 18, 2018 [sic], and has the title “Matter; Why Your DNA Is Still Uncharted Territory.” Where there are differences in wording between the versions, the passages quoted above follow the online version.)

The paper in PLOS Biology co-authored by Thomas Stoeger and mentioned above is:

Stoeger, Thomas, Martin Gerlach, Richard I. Morimoto, and Luís A. Nunes Amaral. “Large-Scale Investigation of the Reasons Why Potentially Important Genes Are Ignored.” PLOS Biology 16, no. 9 (2018): e2006643.

Kealey’s book, praised above, is:

Kealey, Terence. The Economic Laws of Scientific Research. New York: St. Martin’s Press, 1996.

Rationality-Defender Stigler Saw Voting as Irrational, but Did It Anyway

Nobel Prize winner George Stigler contributed to the Public Choice literature and was a staunch defender of rationality. One example would be his paper with Gary Becker, “De Gustibus Non Est Disputandum.” One popular, much discussed conclusion of some public choice theorists is that it is irrational to vote. The argument goes that the marginal effect of one vote is almost always miniscule, so the expected benefit to the voter is equally miniscule. On the other hand, the time and effort it takes to vote are always more than miniscule. So the expected costs of voting exceed the expected benefits. Ergo it is irrational to vote. When I was a graduate student, taking courses in philosophy and economics, and for a couple of years as a post-doctoral fellow, I frequently stopped by the office of the Journal of Political Economy where Stigler was an editor. I believe it was there that I heard Stigler, definitely on an election day, say “Here I go to do something irrational.”

Stigler is well-known for his humorous biting comments. These could be tough on others. But this story shows that they also could be directed at himself.

I do not know if anyone has fully solved the paradox of the irrationality of voting. I guess you would have to say something about how the effects of all good people ceasing to vote would be far from marginal and far from good.

I once mentioned to distinguished Public Choice theorist Dwight Lee that a positive result of the personal benefits of voting being miniscule to a voter, is that the voter was freed from voting their personal narrow self-interest, and could vote their conscience about what served the general good. (Maybe something like what Rawls hoped for behind his “veil of ignorance” in A Theory of Justice.) I believe that Dwight told me that he already published a paper that expressed this positive result, but I never took the time to look for that paper.

Stigler’s Account of Friedman’s “Exhilarating” Conversion of 20 Chicago Economists to Coase’s Theorem

(p. C8) Although he never reached the renown of his lifelong friend Milton Friedman, George Stigler was a founding member of the Chicago school of economics. His charming and readable memoir—really a linked series of vignettes—recounts his time at Chicago, from graduate school to professor.

. . .

Riveting accounts of notable moments in the history of economic thought include the “Coase conversion evening”—a long argument that ended with Friedman convincing 20 economists to embrace a founding theorem of the law and economics movement. “What an exhilarating event,” Stigler recalls. “I lamented afterward that we had not had the clairvoyance to tape it.”

For the full review, see:

Jennifer Burns. “Five Best on Biographies of Economists.” The Wall Street Journal (Saturday, November 4, 2023): C8.

(Note: ellipsis added.)

(Note: the online version of the review has the date November 3, 2023, and has the title “Five Best: Lives of Economists.”)

The book under review is:

Stigler, George J. Memoirs of an Unregulated Economist. New York: Basic Books, Inc., 1988.

Openness to Creative Destruction Released Today

Art Diamond next to sign saying "Congratulations!".
My wife Jeanette decorated the mantelpiece to celebrate the publication of Openness. Our daughter Jenny, and dachshund Fritz, joined the celebration.

Today, June 3, 2019, is the official release date of my book Openness to Creative Destruction: Sustaining Innovative Dynamism.

A couple of weeks ago I heard a thoughtful presentation by Pete Boettke that contrasted the role of economist as scientist and as savior. He plausibly claimed that one of my mentors, George Stigler, defended the economist as scientist.

But as is true of many of us, Stigler was not always consistent. He sometimes said that whether you are a fireman or an incendiary, you need to know how fire works. And I generalize that if you want to be effective at saving the world, you need to know how the world works. Science as a method of tolerant inquiry, and not as a body of unquestionable doctrine, helps you to know how the world works.

So my immodest hope for Openness is both that it advances the science of economics, and that it helps to save the world.

PS: Openness can be purchased from Oxford University Press for 30% off using the discount code ASFLYQ6.

More Evidence for Stigler’s Capture Theory

(p. A15) WASHINGTON — Marilyn B. Tavenner, the former Obama administration official in charge of the rollout of HealthCare.gov, was chosen on Wednesday to be the top lobbyist for the nation’s health insurance industry.
Ms. Tavenner, who stepped down from her federal job in February, will become president and chief executive of America’s Health Insurance Plans, the trade group whose members include Aetna, Anthem, Humana, Kaiser Permanente and many Blue Cross and Blue Shield companies.
As the new voice for insurers, Ms. Tavenner will lead the industry in a time of tumultuous changes and challenges, including delicate negotiations with Congress over the future of the Affordable Care Act.
. . .
The board of America’s Health Insurance Plans unanimously elected Ms. Tavenner at a meeting here on Wednesday, according to Mark B. Ganz, the board chairman, who is also the chief executive of Cambia Health Solutions, based in Portland, Ore.
. . .
Mr. Ganz said that Ms. Tavenner had “the trust and respect of members of Congress from both sides of the aisle.”
Senator John Barrasso, Republican of Wyoming, described the selection in more negative terms. “While millions of Americans are still being hurt by Obamacare’s soaring costs and fewer choices,” he said, “Ms. Tavenner’s appointment shows how the law has created a cozy and profitable relationship for some.”

For the full story, see:
ROBERT PEAR. “Head of Obama’s Health Care Rollout to Lobby for Insurers.” The New York Times (Thurs., JULY 16, 2015): A15.
(Note: ellipses added.)
(Note: the online version of the story has the date JULY 15, 2015.)

Nader Enlists Mises, Hayek, Friedman and Stigler in Critique of Crony Capitalism

(p. A9) Mr. Nader, the consumer crusader who ran for president to the left of Al Gore, is perhaps the last person one would expect to admire a libertarian critique of the corporate state. But in “Unstoppable” he respectfully describes the views of Ludwig Von Mises, Friedrich von Hayek, Milton Friedman, George Stigler and other free-market economists. He praises their distrust of politicians, lobbyists and businessmen who seek to put government power in the service of corporate profit.
Not that the Republican Party is always guided by such thinkers. Mr. Nader neatly describes how corporatist RINOs (Republican In Name Only) co-opt the party’s anti-statist crusaders. “The corporatist Republicans,” he writes, “let the libertarians and conservatives have the paper platforms . . . and then move into office, where they are quick to throw out a welcome mat for Big Business lobbyists with their slush funds.” He cites Adam Smith’s suspicion of regulations that benefit special interests: “Such restraints favor the privileged interests that want to entrench their economic advantages through the force of law.”
These are profound observations and ones that I saw play out while editing the Americas column for this newspaper in the 1980s and ’90s. Mercantilist Latin American businessmen who claimed to cheer market forces often thrived only because of their contacts in government. They reached out to the Journal’s editorial page as allies but were more socialist in practice than some of their left-wing enemies. Little did I suspect that a similar form of mercantilism, or corporate statism, would take root in the U.S. It is a pleasure to see Mr. Nader doing battle against such cozy arrangements.

For the full review, see:
DAVID ASMAN. “BOOKSHELF; Let’s Make a Deal; Ralph Nader’s latest crusade is against the convergence of big business and government power. Let’s hope he succeeds.” The Wall Street Journal (Fri., July 18, 2014): A9.
(Note: ellipsis in original.)
(Note: the online version of the review has the date July 17, 2014, and has the title “BOOKSHELF; Book Review: ‘Unstoppable’ by Ralph Nader; Ralph Nader’s latest crusade is against the convergence of big business and government power. Let’s hope he succeeds.”)

Book under review:
Nader, Ralph. Unstoppable: The Emerging Left-Right Alliance to Dismantle the Corporate State. New York: Nation Books, 2014.

Harry Reid Hires GE Employee to Be His Chief Tax Policy Advisor

The “Capture Theory” associated with scholars George Stigler and Gabriel Kolko says that government regulatory bodies tend to be captured by the companies that they are intended to regulate. Stigler and Kolko would not be surprised by the passage quoted below.

(p. B5) . . . on Jan. 25, Mr. Reid’s office announced that he had appointed Cathy Koch as chief adviser to the majority leader for tax and economic policy. The news release lists Ms. Koch’s admirable and formidable experience in the public sector. “Prior to joining Senator Reid’s office,” the release says, “Koch served as tax chief at the Senate Finance Committee.”

It’s funny, though. The notice left something out. Because immediately before joining Mr. Reid’s office, Ms. Koch wasn’t in government. She was working for a large corporation.
Not just any corporation, but quite possibly the most influential company in America, and one that arguably stands to lose the most if there were any serious tax reform that closed corporate loopholes. Ms. Koch arrives at the senator’s office by way of General Electric.
Yes, General Electric, the company that paid almost no taxes in 2010. Just as the tax reform debate is heating up, Mr. Reid has put in place a person who is extraordinarily positioned to torpedo any tax reform that might draw a dollar out of G.E. — and, by extension, any big corporation.
Omitting her last job from the announcement must have merely been an oversight. By the way, no rules prevent Ms. Koch from meeting with G.E. or working on issues that would affect the company.

For the full story, see:
JESSE EISINGER, ProPublica. “A Revolving Door in Washington With Spin, but Less Visibility.” The New York Times (Thurs., February 21, 2013): B5.
(Note: ellipsis added.)
(Note: the online version of the story has the date February 20, 2013.)

Behavioral Economics Does Not Undermine Capitalism

thinkingfastandslowBK2012-06-21.jpg

Source of book image: http://www.brainpickings.org/wp-content/uploads/2011/10/thinkingfastandslow.jpg

Daniel Kahneman first gained fame in economics through research with Tversky in which they showed that some of economists’ assumptions about human rationality do not always hold true.
Kahneman, whose discipline is psychology, went on to win the Nobel Prize in economics, sharing the prize with Vernon Smith. (Since the Prize is not normally awarded posthumously, Tversky was not a candidate.)
I have always thought that ultimately there should be only one unified science of human behavior—not claims that are “true” in economics and other claims that are “true” in psychology. (I even thought of minoring in psychology in college, before I realized that the price of minoring included taking time-intensive lab courses where you watched rats run through mazes.)
But I don’t think the implications of current work in behavioral economics are as clear as has often been asserted.
Some important results in economics do not depend on strong claims of rationality. For instance, the most important “law” in economics is the law of demand, and that law is due to human constraints more than to human rationality. Gary Becker, early in his career, wrote an interesting paper in which he showed that the law of demand could also be derived from habitual and random behavior. (I remember in conversation, George Stigler saying that he did not like this paper by Becker, because it did not hone closely to the rationality assumption that Stigler and Becker defended in their “De Gustibus” article.)
The latest book by Kahneman is rich and stimulating. It mainly consists of cataloging the names of, and evidence for, a host of biases and errors that humans make in thinking. But that does not mean we cannot choose to be more rational when it matters. Kahneman believes that there is a conscious System 2 that can over-ride the unconscious System 1. In fact, part of his motive for cataloging bias and irrationality is precisely so that we can be aware, and over-ride when it matters.
Sometimes it is claimed, as for instance in a Nova episode on PBS, that bias and irrationality were the main reasons for the financial crisis of 2008. I believe the more important causes were policy mistakes, like Clinton and Congress pressuring Fannie Mae and Freddie Mac to make home loans to those who did not have the resources to repay them; and past government bailouts encouraging finance firms to take greater risks. And the length and depth of the crisis were increased by government stimulus and bailout programs. If instead, long-term cuts had been made in taxes, entrepreneurs would have had more of the resources they need to create start-ups that would have stimulated growth and reduced unemployment.
More broadly, aspects of behavioral economics mentioned, but not emphasized, by Kahneman, can actually strengthen the underpinnings for the case in favor of entrepreneurial capitalism. Entrepreneurs may be more successful when they are allowed to make use of informal knowledge that would not be classified as “rational” in the usual sense. (I discuss this some in my forthcoming paper, “The Epistemology of Entrepreneurship.”)
Still, there are some useful and important examples and discussions in Kahneman’s book. In the next several weeks, I will be quoting some of these.

Book discussed:
Kahneman, Daniel. Thinking, Fast and Slow. New York: Farrar, Straus and Giroux, 2011.

The Becker article mentioned above is:
Becker, Gary S. “Irrational Behavior and Economic Theory.” Journal of Political Economy 70, no. 1 (Feb. 1962): 1-13.

The Stigler-Becker article mentioned above is:
Stigler, George J., and Gary S. Becker. “De Gustibus Non Est Disputandum.” American Economic Review 67, no. 2 (March 1977): 76-90.