The National Association of Scholars welcomes Secretary Scott Pruitt’s announcement of a proposed rule change at the Environmental Protection Agency. The new rule requires that “When promulgating significant regulatory actions, the Agency shall ensure that dose response data and models underlying pivotal regulatory science are publicly available in a manner sufficient for independent validation.” The NAS has long been concerned about politicized distortions of dose-response science, and our just-released report The Irreproducibility Crisis: Causes, Consequences, and the Road to Reform urges that the Federal government require that only properly reproducible science be used as a basis for regulatory action. We are delighted that Secretary Pruitt’s proposed action addresses two of our priorities. We are also gratified that Secretary Pruitt has provided proper flexibility in the application of this new principle. The proposed rule allows the EPA Administrator to grant exemptions on a case-by-case basis.
The NAS urges Federal agency heads to take steps similar to Secretary Pruitt’s initiative. Legislators must also act to ensure that Federal policy is based on reproducible science. We recommend five changes in the way government funds and regulates science.
- Government granting agencies should fund more studies of publication bias, more re-examinations of the statistical operations underlying older studies, and more studies of the effects of groupthink. These agencies should prioritize funding for these studies. They should also prioritize funding for research on improving experimental design techniques.
- Federal legislators should dedicate funding for programs to broaden statistical literacy in primary, secondary, and post-secondary education. They should fund basic classes in statistical literacy for all high school and college students and advanced statistics education for students specializing in the sciences and social sciences.
- Legislators should fund infrastructure to allow scientists to make their work completely reproducible. These should include the creation of standardized descriptions of scientific materials and procedures, standardized statistics programs, and standardized archival formats. Policymakers should fund the infrastructure to create born-open data, facilitate pre-registered research protocols, and fund journals dedicated to negative results.
- Policymakers should work with reform-minded professionals in each discipline to draft policies to condition government funding on scientists adopting best existing practices to deal with the reproducibility crisis. They should institute regulations that require grant-seekers to adopt standardized materials and techniques, born open data, and preregistered research protocols. In a world of scarce funding, policymakers should prioritize funding for science done properly.
- Policymakers should require all new science regulations be founded on research that meets strict reproducibility standards. Congress should pass a Reproducible Science Reform Act that requires all future regulations to be based on such standards. Policymakers should not make policy informed by scientific research unless they are sure that research can be reproduced.
Secretary Pruitt’s commendable initiative complements a great deal of excellent work by scientists, businesses, and private foundations to improve scientific reproducibility. Further work by the Federal government can and must complement the work Secretary Pruitt has begun.
Image Credit: Tim Evanson
