Zeke Hausfather, Research Scientist at Berkeley Earth, wrote a comment to dispute NAS’s rendition of the debate between John Bates and Tom Karl about the reproducibility of National Oceanic and Atmospheric Administration (NOAA) climate data. Irreproducibility Crisis co-author David Randall replies briefly.
I welcome Mr. Hausfather’s civil critique of The Irreproducibility Crisis’s discussion of the John Bates/Tom Karl debate about proper archiving procedures at NOAA.
The Irreproducibility Crisis drew on John Bates’ February 4, 2017 discussion of the issues at Climate Etc. The crucial paragraph of Bates’ original article is this:
In 2013, prior to the Karl study, the National Climatic Data Center [NCDC, now the NOAA National Centers for Environmental Information (NCEI)] had just adopted much improved processes for formal review of Climate Data Records, a process I formulated [link]. The land temperature dataset used in the Karl study had never been processed through the station adjustment software before, which led me to believe something was amiss. When I pressed the co-authors, they said they had decided not to archive the dataset, but did not defend the decision. One of the co-authors said there were ‘some decisions [he was] not happy with’. The data used in the K15 paper were only made available through a web site, not in digital form, and lacking proper versioning and any notice that they were research and not operational data. I was dumbstruck that Tom Karl, the NCEI Director in charge of NOAA’s climate data archive, would not follow the policy of his own Agency nor the guidelines in Science magazine for dataset archival and documentation.
Mr. Hausfather’s critique draws significantly upon the response he drafted the next day, on February 5, 2017. Readers interested in the details of the debate should also see the counter-responses to Mr. Hausfather, and other critics, posted on Climate Etc. on February 6, 2017.
Mr. Hausfather’s reply does not address “the specific technical details of the data archiving procedures associated with the Karl et al paper,” which are the nub of the matter. The construction of datasets like those used in climate science is a complex process, and Karl failed to follow his agency's established procedures for documentation and quality control, which had been instituted due to past problems with data quality. It is not at all clear that the data now available on the FTP site to which Mr. Hausfather links are sufficient to allow reproduction of Karl et al. 2015.
The NAS shares Dr. Hausfather’s concern that climate science should be properly reproducible. It remains unclear to us whether Karl’s 2015 paper met that standard.
Image Credit: David Randall
