Among Stanford University's palm trees and sandstone arches—the picture of ivory-tower—there's going to be a center dedicated to chipping away that ivory, The Economist reports. The university plans to open a Meta-Research Innovation Center: a center for the study of bad science.
The center will take aim at biomedical studies that are poorly designed, are redundant, weren't published because they report negative results, and/or independent scientists can't reproduce. The idea is that an enormous number of published studies showing that Drug X cures Disease Y, or that Gene A is related to Condition B, are actually incorrect. Drug X doesn't cure Disease Y, or isn't really better at curing Disease Y than Drug Z, which has been around forever. Nevertheless, because of poor statistical analysis or other problems, scientific journals publish these untrue studies. Doctors then read those studies and use them to help decide how to treat their patients.
Epidemiologist John Ioannidis will lead the Meta-Research Innovation Center, The Economist reports. Ioannidis is a well-known figure in the study of bad science. He's perhaps best known for publishing a study in 2005 in which he developed a statistical model of how often published papers are wrong. In a 2010 profile of Ioannidis, The Atlantic summarized the findings:
His model predicted, in different fields of medical research, rates of wrongness roughly corresponding to the observed rates at which findings were later convincingly refuted: 80 percent of non-randomized studies (by far the most common type) turn out to be wrong, as do 25 percent of supposedly gold-standard randomized trials, and as much as 10 percent of the platinum-standard large randomized trials. . . . 'You can question some of the details of John's calculations, but it's hard to argue that the essential ideas aren't absolutely correct,' says Doug Altman, an Oxford University researcher who directs the Centre for Statistics in Medicine.
Several scientists and journalists have tried to call attention to these problems in the medical literature. (They exist in other fields of science besides medicine, of course. However, people often focus on medicine because it affects how doctors treat patients.) The problems still persist. This new effort from Stanford will try to further develop the field of examining bad science, The Economist reports. In addition, the center will examine whether doctors even pay attention to projects that try to reproduce the findings from famous studies. In other words, the Meta-Research Innovation Center will investigate the impact of other centers dedicated to finding bad science. Meta indeed.
