In the Actual Messy Real-World, RCTs Can Be a Fool’s Gold Standard

George Stigler used to emphasize that when we compare markets to the government we often make an unfair comparison. We compare the ideal theoretical government, with the actual messy real-world market. We do the same thing in comparing methods of reaching medical truth. In this analogy the role of the government is played by double-blind randomized clinical trials (RCTs). In ideal theory, RCTs are the “gold standard.” The story quoted from below, shows that a lot can go wrong when RCTs are conducted in the actual messy real-world.

Note that the flaws in the RCT were NOT caught in the peer-review process, but were caught by “online sleuths and physicians,” practitioners with a sense of urgency and mission.

(p. D2) Early this year, a medical journal article caught the attention of cancer patients and doctors worldwide because of its extraordinary conclusion. Simply changing the time of day that immunotherapy was administered appeared to produce a stunning benefit for lung cancer patients.

Those who received IV infusions in the morning had their cancer kept at bay for twice as long as those who got it in the afternoon, according to the results from a clinical trial in China and published in the journal Nature Medicine in February [2026]. The study also reported that the patients lived nearly twice as long.

. . .

But on Wednesday [June 24, 2026], Nature Medicine retracted the study, citing a list of inconsistencies and irregularities in the trial’s design and results.

. . .

Among the issues that the journal cited in its retraction notice: Records that were supposed to be locked before the study started were changed midway through. There were discrepancies between the Chinese version of the study’s plan and the translated version. Every patient remained treated and tracked in the study’s first year, and no one dropped out because of side effects — highly unusual in an oncology study. And unusual patterns were found in the timing of follow-up scans.

“Due to the amount and nature of the problems identified, the editors no longer have confidence in the integrity of the results,” the journal said.

. . .

China’s critics often question the reliability of its biomedical research. Experts said that, similar to studies in the United States, China’s research output spans a wide range in quality: Some Chinese scientists run their studies at the most meticulous standards. Others were said to cut corners.

. . .

The study had enrolled 210 patients with advanced lung cancer at Hunan Cancer Hospital in Changsha, a city in south-central China. Patients were randomly assigned to receive infusions of an immunotherapy — Merck’s blockbuster drug Keytruda or Tyvyt, which is not approved in the United States — either before or after 3 p.m.

The study reported that tumors did not progress for 11 months in patients who received the earlier infusions, compared with six months for those given the later infusions. Patients receiving the earlier infusions lived for 28 months, compared with 17 months for those infused later in the day.

. . .

Dr. Anil Makam, an epidemiologist and health services researcher at the University of California, San Francisco, said such a drastic benefit, if real, would have prompted infusion clinics to overhaul staffing and scheduling to shift appointments to earlier in the day.

“If we believed the effects, it would be malpractice not to,” he said.

But within days after the study’s publication in Nature Medicine, online sleuths and physicians like Dr. Makam began raising concerns on social media and in blog posts. Less than three weeks after the study was published, the journal published an editor’s note saying it was investigating the issues.

In a statement on Thursday [June 25, 2026], Dr. João Monteiro, chief editor of Nature Medicine, which is published by Springer Nature, said, “We are grateful to the research community for bringing these concerns to our attention.”

For the full article, see:

Rebecca Robbins. “‘Too Good to Be True’: Chinese Study on Cancer Therapy Is Retracted.” The New York Times (Tues., June 30, 2026): B3.

(Note: ellipses, and bracketed dates, added.)

(Note: the online version of the Houreld article was updated June 25, 2026, and has the title “‘Too Good to Be True’: A Chinese Study on Timing Cancer Therapy Is Retracted.” Where there is a slight difference in wording, the quotations above follow the print version.)

Leave a Reply

Your email address will not be published. Required fields are marked *