The Establishment Defends Irreproducibility

David Randall

A recent article by Robinson Meyer in The Atlantic continues the drumbeat of opposition by the scientific establishment to the EPA’s proposed rule Strengthening Transparency in Regulatory Science. (The NAS has provided a public comment in favor of that rule.) Meyer quotes various scientists as saying that forbidding the use of un-transparent data will cripple public health regulation. In making this argument, both Meyer and the scientists he quote fail to mention several previous points of contention. Readers interested in this issue should find these points worth reviewing.

Meyer does not mention the possibility of de-identifying previous research, rendering it anonymous, and thus preserving the anonymity of participants. There is legitimate debate as to how much such research would cost: the EPA claims that it would cost more than $250 million a year—although this may be an inflated estimate, provided to deter would-be reformers. There is also the question of how effective de-anonymizing techniques can be, in an age of massive computational power.

Americans should debate the cost of anonymization, its priority within EPA spending, and how best to de-anonymize data. The NAS favors a substantial shift of resources by the EPA to anonymize data, using the best available techniques, sufficient in scale to allow for the de-identification within 10 years of the majority of research used by the EPA to inform its regulations. We urge the scientific establishment to join us in this call, which will provide the EPA properly reproducible scientific data.

Meyer also does not mention the possibility that scientists can use publicly accessible data to study public health issues—as some have already begun to do.

“This is a very highly contentious political climate, and we are taking the extra step to be as transparent as we can be,” says biostatistician Francesca Dominici of the Harvard T. H. Chan School of Public Health in Boston, who led the studies. In one, published in The New England Journal of Medicine in June 2017, she and her colleagues used publicly accessible air pollution data and records compiled by the federal government’s Medicare health insurance program to show that even modest pollution reductions could save more than 10,000 lives per year. In another, published in JAMA last December, they linked short-term exposure to air pollution levels below current limits to premature death among the elderly.

Stricter transparency regulations may annoy scientists, and add marginally greater costs to their procedure—but the contention that science will be crippled depends on the presumption that scientific ingenuity cannot find workable ways to meet strict transparency standards. Dominici’s research already provides evidence that this presumption is incorrect.

The NAS urges the scientific establishment to take account of these issues and developments as it critiques the EPA’s proposed rule. And we repeat our recommendation to the EPA that it adopt strict transparency regulations—with redoubled confidence that American scientists can produce solid research that meets these regulatory standards.

  • Share

Most Commented

October 29, 2024

1.

The Looming Irrelevance of Middle East Study Centers

Today’s Middle Eastern Studies Centers are facing a crisis due to the winds of change in the Middle East and their own ideological echo chamber....

November 19, 2024

2.

Lee Zeldin Should Reform EPA Science Policy

NAS welcomes the nomination of Congressmen Lee Zeldin to lead the Environmental Protection Agency....

November 20, 2024

3.

NAS Welcomes Administrator McMahon's Nomination to Serve as Education Secretary

With McMahon, the new administration has a chance to drastically slim down and depoliticize the Education Department....

Most Read

May 15, 2015

1.

Where Did We Get the Idea That Only White People Can Be Racist?

A look at the double standard that has arisen regarding racism, illustrated recently by the reaction to a black professor's biased comments on Twitter....

October 12, 2010

2.

Ask a Scholar: What is the True Definition of Latino?

What does it mean to be Latino? Are only Latin American people Latino, or does the term apply to anyone whose language derived from Latin?...

May 26, 2010

3.

10 Reasons Not to Go to College

A sampling of arguments for the idea that college may not be for everyone....