Randomized Controlled Trials Can Obscure “Nuances and Complexities”

(p. A17) You’ve probably heard of the “gold standard”—randomized controlled trials—for evaluating new pharmaceutical therapies, including for Covid-19. Many treatments that showed promise in other studies have turned up muddy results in randomized controlled trials. But that doesn’t mean they’re necessarily ineffective. Doctors and regulators need to consider the totality of medical evidence when treating patients.

. . .

“Randomized trials for some purposes is the gold standard, but only for some purposes,” Harvard’s Donald Berwick, a former health adviser to President Barack Obama, said in an interview with GNS Health Care CEO Colin Hill in 2013. “Context does matter. We’re learning in a very messy world, and the context that neatens up that world may make it hard to know how to manage in the real world.”

. . .

As Thomas Frieden, who directed the Centers for Disease Control and Prevention under Mr. Obama, wrote in a 2017 New England Journal of Medicine article: “Elevating RCTs at the expense of other potentially highly valuable sources of data is counterproductive.” Such limitations affect their use for “urgent health issues, such as infectious disease outbreaks.” He added: “No study design is flawless, and conflicting findings can emerge from all types of studies.”

. . .

Some experts have dismissed the antimalarial hydroxychloroquine, or HCQ, even though more than a dozen observational studies have found it beneficial. A retrospective observational study of Covid-infected nursing-home residents in France, for instance, found those treated with HCQ and azithromycin were 40% less likely to die.

But a few randomized controlled trials found no benefit. A Spanish randomized trial of HCQ for prophylaxis found it didn’t reduce risk of illness among a large group of people exposed in nursing homes, households and health-care settings. Yet two-thirds of the subjects “reported routine use of masks at the time of exposure,” so they were probably less likely to be infected. Nursing-home residents, who may be less likely to wear masks, were 50% less likely to become sick if they took HCQ. But this finding was statistically insignificant, because the trial included only 293 residents.

. . .

Another problem with Covid-19 randomized trials: Patients at different stages of an illness are often assigned the same dosage. Trials don’t reveal differences in how patients respond to a drug at different dosages or illness severity.

Observational studies can do so. Consider a large study by the Mayo Clinic, which found no overall benefit among patients who received a higher-antibody convalescent plasma versus a lower one. Yet the researchers reported a 37% reduction in mortality among patients under 80 who weren’t on a ventilator and received a high-antibody plasma within three days of hospitalizations.

A randomized trial might have obscured these nuances and complexities, denying doctors important information about treatment options. Randomized controlled trials can yield important insights, but it is a medical mistake and a disservice to patients to dismiss other types of evidence.