Last November Nature published an editorial opposing the Environment Protection Agency's proposed rule, "Strengthening Transparency in Regulatory Science." S. Stanley Young (Director, National Association of Scholars' Shifting Sands Project), Warren Kindzierski (University of Alberta), and David Randall (Director of Research, National Association of Scholars) sent in a letter stating briefly the reasons that Nature ought to support the rule—in the interest of reproducible science. Nature has declined to publish the letter. We post it here.
Dear Editor:
Scientists frequently discover that an entire discipline has proceeded down a blind alley. Begley and Ellis gave experimental biology a wake-up call (see Nature 483, 531–533; 2012). Psychology is sorting out its own replication crisis. A Nature survey of ~1500 scientists revealed that ~90% considered science to be suffering a systemic crisis (Nature 533, 452–454; 2016). Environmental epidemiology merits re-examination as much as any other discipline. Scientists can only correct course by re-examining their data sets; public access to data is essential.
--
Researchers should unite in support of the US Environmental Protection Agency’s proposed requirement for data access, not against it (see Nature 575 415;2019). The US Food and Drug Administration and the European Medicines Agency both require their access to clinical and observational data. Multiple journals require authors to make their data available: Nature, Science, Royal Society B, PNAS, etc. The entire research output of experimental biology was called into question (see Nature 483, 531–533;2012) when it was reported that 47 out of 53 experimental biology claims could not be replicated; many research labs could not replicate their own findings. Nature’s editors claim that the science is settled concerning the health effects of air quality, but that claim relies on old research long since subjected to substantial and unrebutted criticisms, and neglects to account for several recent, large, well-conducted negative studies (see Critical Reviews in Toxicology 49 85-94;2019 and references cited therein). In the CRT study, a meta-analysis using 34 base papers is examined for air quality components (particulate matter (PM10 and PM2.5), carbon monoxide, nitrogen dioxide, sulphur dioxide, and ozone) and induction of heart attacks. The number of statistical tests possible in the base papers were large, median = 12,288 (interquartile range = 2496 − 58,368), so there is ample opportunity for selective reporting. For all six air components, the shapes of p-value plots were consistent with the possibility of “p-hacking”—analysis manipulation to obtain small p-values. The extensive heterogeneity of findings strongly suggests that the air quality literature may have been affected by different aspects of the replication crisis. In these circumstances, the EPA’s call for open data access seems eminently reasonable and in keeping with a key requirement of science. Nature should be at the forefront of the fight for fully reproducible science, not opposing it.
References (3)
Researchers must unite against US environment agency’s attack on scientific evidence.
Nature 575 (21 November 2019) 415, doi:10.1038/d41586-019-03526-z.
Begley CG, Ellis LM. 2012. Raise standards for preclinical cancer research. Nature 483, 531–533, doi:10.1038/483531a.
Young SS, Kindzierski KB. 2019. Evaluation of a meta-analysis of air quality and heart attacks, a case study, Critical Reviews in Toxicology, doi:10.1080/10408444.2019.1576587 See also arXiv:1904.01676.
Image: Stadtbibliothek, Stuttgart-Mitte, Stuttgart, Baden-Württemberg, Deutschland by Tobias Fischer on Unsplash
