Donate Join
×
Donate  /  Join
Reproducible Policy Act

The National Association of Scholars upholds the standards of a liberal arts education that fosters intellectual freedom, searches for the truth, and promotes virtuous citizenship.

Introduction

The Federal government should not impose regulations based on research that is not publicly accessible, and that therefore is not reproducible. The public can only hold government accountable if it knows the data which informs government action. The government’s willingness to accept research that is not publicly accessible also provides an incentive to politically motivated researchers to use research with hidden data so as to change government policy.

Our model bill 1) requires federal agencies to use only publicly accessible research which (in appropriate disciplines) meets Good Laboratory Practice Standards in the most important regulatory actions and scientific inventories; 2) requires federal agencies to use professional literature assessments to judge the quality of bodies of research informing these most important regulatory actions and scientific inventories; 3) requires federal agencies to fund reviews of existing regulatory actions and scientific inventories and rescind regulations that do not use only publicly accessible research; and 4) requires federal agencies to establish a quantitative measure of how often scientific research has been confirmed by replication studies and apply that quantitative measure into their scientific inventories and assessments of weight of evidence.

Our model bill refers to the {Environmental Protection Agency}. The bill should be adjusted by policymakers to refer to some or all individual federal agencies funding science grants. Similar references to the {Administrator of the Environmental Protection Agency}, the {Executive Branch Office}, the {Senate Committee}, and the {House Committee} also should be adjusted appropriately by policymakers.


Model Legislative Text

  1. Publicly Accessible Research. The {Environmental Protection Agency} after {Month Day, Year} may include only publicly accessible research in professional assessments, administrative processes, significant regulatory actions, highly influential scientific assessments, and the {Environmental Protection Agency}’s Science Inventory.
  2. Good Laboratory Practice Standards. The {Environmental Protection Agency} after {Month Day, Year} may include only research that meets Good Laboratory Practice Standards, as described in 40 CFR 160, in professional assessments, administrative processes, significant regulatory actions, highly influential scientific assessments, and the {Environmental Protection Agency}’s Science Inventory.
  3. Professional Literature Assessments. The {Environmental Protection Agency} after {Month Day, Year} must include a professional literature assessment for all significant regulatory actions and highly influential scientific assessments.
  4. Review. The {Administrator of the Environmental Protection Agency} shall establish a commission within one calendar year of {Month Day, Year}, which shall complete its work within three calendar years of {Month Day, Year}, to review all significant regulatory actions and highly influential scientific assessments promulgated before {Month Day, Year} and assess whether each significant regulatory action or highly influential scientific assessment still possesses sufficient scientific justification.
    1. The {Environmental Protection Agency} shall dedicate {5%} of its funding to funding the activities described in subsection (4).
    2. The {Environmental Protection Agency} shall by {Month Day, Year} make publicly available on its website the regulations guiding the commission described in subsection (4), all information discovered by the commission described in subsection (4), and the conclusions of the commission described in subsection (4).
  5. Recission. The {Administrator of the Environmental Protection Agency} shall within five calendar years of {Month Day, Year} rescind all significant regulatory actions and highly influential scientific assessments that the commission described in subsection (4) determines no longer possess sufficient scientific justification.
  6. Quantitative Assessments. The {Administrator of the Environmental Protection Agency} shall establish a quantitative measure of how often scientific research has been confirmed by replication studies, apply that quantitative measure to all research articles in its Health & Environment Research Online (HERO) database, and incorporate this quantitative assessment into its assessments of scientific weight of evidence.
  7. Waivers. The {Administrator of the Environmental Protection Agency} may provide an individual waiver for subsections (1-3) if:
    1. on an annual basis, starting no later than {Month Day, Year}, he submits to the {Executive Branch Office}, the {Senate Committee}, and the {House Committee} a report cataloguing, describing, and explaining each waiver granted under his authority in subsection (4); and
    2. he creates within one year of {Month Day, Year} a set and detailed procedure to allow private individuals and organizations to challenge each waiver granted under his authority in subsections (1-3).
  8. Severability. If any provision of this chapter, or the application of any provision to any person or circumstance, is held to be invalid, the remainder of this chapter and the application of its provisions to any other person or circumstance shall not be affected thereby.
  9. Definitions.
    1. “Publicly accessible research” shall be defined as research whose registered report (including protocols), research data, associated protocols, computer codes, data analysis scripts, recorded factual materials, and statistical analyses are archived on an online digital repository in a manner sufficient for continuing independent inspection, replication, reproduction, and/or verification. This online digital repository shall have archival and accessibility capacities at least equal to those possessed by the Open Science Framework on December 31, {Year}.
    2. “Professional assessments” shall be defined as environmental data, information quality, research data, and all other professional assessments.
    3. “Administrative processes” shall be defined as action development, communications with the public, enforcement activities, guidance documents, individual party adjudications, non-binding regulatory determinations, non-regulatory actions, peer review, permit proceedings, policy statements, professional development, promulgations, proposed rules, regulatory actions, regulatory decisions, regulatory proposals, risk assessments, rulemakings, site-specific permitting actions, work products, and all other agency actions with precedent-setting influence on future actions.
    4. “Significant regulatory action” shall be defined as any regulatory action that is likely to result in a rule that may:
      1. have an annual effect on the economy of $100 million or more or adversely affect in a material way the economy, a sector of the economy, productivity, competition, jobs, the environment, public health or safety, or State, local, or tribal governments or communities;
      2. create a serious inconsistency or otherwise interfere with an action taken or planned by another agency;
      3. materially alter the budgetary impact of entitlements, grants, user fees, or loan programs or the rights and obligations of recipients thereof; or
      4. raise novel legal or policy issues arising out of legal mandates, the President’s priorities, or the principles set forth in Executive Order 12866.
    5. “Highly influential scientific assessment” shall be defined as any scientific assessment, as determined by the OMB Office of Information and Regulatory Affairs Administrator or the {Administrator of the Environmental Protection Agency}, which could have a potential impact of more than $500 million in any one year on either the public or private sector, or whose dissemination is novel, controversial, precedent-setting, or has significant interagency interest.
    6. “Meta-analysis” shall be defined as the combining of evidence from independent studies using appropriate statistical methods.
    7. “Systematic review” shall be defined as research that aims to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question, using explicit methods aimed at minimizing bias, in order to produce more reliable findings that can be used to inform decision making.
    8. “Publication bias study” shall be defined as a peer-reviewed study of the effects upon the professional literature on a specific scientific question or an entire discipline of selective (non-)publication, non-dissemination, delayed publication, misrepresentation, and misinterpretation of specific scientific conclusions or whole studies based on the nature and direction of the results.
    9. “Professional literature assessment” shall be defined as:
      1. where available, at least one meta-analysis or systematic review, which must also be publicly accessible research; and
      2. where available, at least one publication bias study, published by different authors from the authors of the meta-analysis or systematic review, which must also be publicly accessible research, on the effect of publication bias on the scientific research under consideration; or
      3. in the absence of either a meta-analysis, systematic review, or a publication bias study, explicit assessment and professional avowal of the unbiased validity of the scientific research under consideration.
    10. “Replication studies” shall mean studies for which:
      1. any outcome would be considered diagnostic evidence about a claim from prior research, such that outcomes consistent with a prior claim would increase confidence in the claim, and outcomes inconsistent with a prior claim would decrease confidence in the claim; and
      2. the subject matter and the procedures meet best professional standards in their attempt to replicate the subject matter and the procedures of the prior research.