ClearFactAll topics

Does increased use of AI in scientific research worsen the reproducibility crisis in academia?

3 peer-reviewed studies for, 3 against

Key takeaways

Stronger evidence: For

Verdict

The evidence indicates that increased use of AI in scientific research is currently worsening the reproducibility crisis on net, even though AI also provides powerful tools that could eventually improve reproducibility if standards and adoption catch up. On the "for" side, Kapoor and Narayanan (Patterns, 2023) systematically reviewed 294 ML-in-science papers across 17 fields and documented pervasive data leakage that produces inflated performance estimates failing to replicate on independent data. McDermott et al. (Science Translational Medicine, 2021) audited ML for health research and found very few studies released full code, data, and models, with frequent failures on re-execution attempts. Day (The Professional Geographer, 2023) showed ChatGPT systematically fabricates plausible-looking but fake academic citations, eroding the verifiability foundations of reproducibility. On the "against" side, Heil et al. (Nature Methods, 2021) showed that bronze/silver/gold reproducibility standards for ML in life sciences are feasible with existing tooling — making the problem solvable rather than intrinsic. Yang, Youyou, and Uzzi (PNAS, 2020) demonstrated that ML models can predict replicability of published findings with accuracy comparable to expert prediction markets, offering a scalable screening layer. Pellegrina and Helmy (Frontiers in Artificial Intelligence, 2025) reviewed AI tools now used to detect image duplication, statistical inconsistencies, fake references, and undisclosed AI-generated text in manuscripts. Putting these together: the "against" papers show AI can improve reproducibility under disciplined deployment, but the "for" papers show what is actually happening at scale right now — leakage in 294 papers across 17 fields, irreproducible ML health models, and LLM-fabricated citations entering the literature. The negative effects are realized; the positive effects are largely potential and contingent on adoption that has not yet occurred broadly. The net direction in the present period is therefore worse, not better. The "for" position prevails for the question as asked, with the important caveat that the "against" tools represent the most credible path out of the crisis if the field commits to them.

Strongest study on each side

Ranked by citation count.

Strongest study for

Leakage and the reproducibility crisis in machine-learning-based science (Patterns, 2023)

Sayash Kapoor and Arvind Narayanan

Kapoor and Narayanan systematically reviewed ML-based scientific work and identified pervasive data leakage — train/test contamination, temporal leakage, sampling bias, and others — affecting 294 papers across 17 scientific fields. Leakage produces inflated performance estimates that fail to hold up on independent data, undermining the validity of published findings. The authors argue that the current ML-in-science boom is creating a reproducibility crisis on top of the one that already existed in fields like psychology and biomedicine.

Strongest study against

AI for scientific integrity: detecting ethical breaches, errors, and misconduct in manuscripts (Frontiers in Artificial Intelligence, 2025)

Diogo Pellegrina and Mohamed Helmy · 18 citations

Pellegrina and Helmy reviewed AI-driven tools now in use across scientific publishing — image-duplication detectors, statistical-inconsistency checkers (e.g., GRIM, statcheck-style tools), reference-validity scanners, and AI-generated-text detectors — and document measurable detection rates against benchmark datasets of known errors and manipulations. They conclude that AI is becoming a scalable quality-control layer for the literature, catching reproducibility-relevant errors at volumes humans cannot match, and that careful deployment of these tools is improving, not degrading, scientific integrity.

Open the interactive page →

The evidence

For (3)

Leakage and the reproducibility crisis in machine-learning-based science (Patterns, 2023)

Sayash Kapoor and Arvind Narayanan

Kapoor and Narayanan systematically reviewed ML-based scientific work and identified pervasive data leakage — train/test contamination, temporal leakage, sampling bias, and others — affecting 294 papers across 17 scientific fields. Leakage produces inflated performance estimates that fail to hold up on independent data, undermining the validity of published findings. The authors argue that the current ML-in-science boom is creating a reproducibility crisis on top of the one that already existed in fields like psychology and biomedicine.

A Preliminary Investigation of Fake Peer-Reviewed Citations and References Generated by ChatGPT (The Professional Geographer, 2023)

Terence Day

Day asked ChatGPT to generate references on geography topics and verified each citation against bibliographic databases. The chatbot fabricated a majority of the references — many with plausible author lists, journal names, and DOIs that did not correspond to any real publication — while presenting them as authentic. The result demonstrates that LLM-assisted scientific writing can introduce hallucinated citations that pollute the literature, eroding the verifiability foundations on which reproducibility depends.

Reproducibility in machine learning for health research: Still a ways to go (Science Translational Medicine, 2021)

Matthew B. A. McDermott, Shirly Wang et al.

McDermott, Wang, Marinsek, Ranganath, Foschini, and Ghassemi audited ML for health research and found that very few studies released full code, data, and trained models, and that even nominally reproducible papers frequently failed when independent groups attempted re-execution. The authors documented systematic underreporting of preprocessing, hyperparameter selection, and dataset splits, concluding that ML adoption in clinical research has so far amplified rather than mitigated the reproducibility problem in medicine.

Against (3)

AI for scientific integrity: detecting ethical breaches, errors, and misconduct in manuscripts (Frontiers in Artificial Intelligence, 2025)

Diogo Pellegrina and Mohamed Helmy · 18 citations

Pellegrina and Helmy reviewed AI-driven tools now in use across scientific publishing — image-duplication detectors, statistical-inconsistency checkers (e.g., GRIM, statcheck-style tools), reference-validity scanners, and AI-generated-text detectors — and document measurable detection rates against benchmark datasets of known errors and manipulations. They conclude that AI is becoming a scalable quality-control layer for the literature, catching reproducibility-relevant errors at volumes humans cannot match, and that careful deployment of these tools is improving, not degrading, scientific integrity.

Reproducibility standards for machine learning in the life sciences (Nature Methods, 2021)

Benjamin J. Heil, Michael M. Hoffman et al.

Heil, Hoffman, Markowetz, Lee, Greene, and Hicks proposed concrete bronze, silver, and gold reproducibility standards for ML in life-sciences research and demonstrated that meeting them is feasible with existing tools — version-controlled code, containerized environments, deposited data, and seeded random states. They argue that the reproducibility issues now visible in ML-based science are addressable with disciplined practice, and that adopting these standards positions AI to improve rather than degrade reproducibility relative to traditional bench science.

Estimating the deep replicability of scientific findings using human and artificial intelligence (PNAS, 2020)

Yang Yang, Wu Youyou et al.

Yang, Youyou, and Uzzi trained machine-learning models on the texts of hundreds of manually replicated studies and showed that AI can predict whether published findings will replicate with accuracy comparable to expert prediction markets, at far lower cost. They argue that AI provides a scalable screening layer for journals, funders, and authors that could meaningfully reduce the prevalence of non-replicable findings entering the literature — a direct example of AI mitigating the reproducibility crisis rather than worsening it.

The debate

Lisa: Kapoor and Narayanan in Patterns systematically reviewed ML-based science across 17 fields and found data leakage in 294 papers — train/test contamination, temporal leakage, sampling bias. The result is inflated performance that does not hold up on independent data. That is the reproducibility crisis being amplified, not solved.

James: That documents a real problem, but the same field is producing the fix. Heil and colleagues in Nature Methods proposed bronze, silver, and gold reproducibility standards for ML in life sciences — version-controlled code, containerized environments, deposited data, seeded random states — and showed they are achievable with current tools. The crisis-side findings are a snapshot of bad practice, not a verdict on AI in science.

Lisa: Standards exist, but adoption is the question. McDermott and colleagues in Science Translational Medicine audited ML for health research and found very few studies actually released code, data, and models. Even when they did, independent re-execution often failed. The standards Heil proposes are not what the median ML-health paper currently meets.

James: Agreed on the gap, but the trajectory matters too. Yang, Youyou, and Uzzi in PNAS trained models that predict whether findings will replicate at accuracy comparable to expert prediction markets, at far lower cost. That is exactly the kind of scalable screening the field needs. AI is not just a contributor to the problem — it is one of the most credible solutions on the table.

Lisa: A predictor is useful, but predicting replication failure does not prevent it. Day in The Professional Geographer asked ChatGPT to generate references and most were fabricated — plausible authors, plausible journals, plausible DOIs, none of them real. LLM-assisted writing is now polluting bibliographies in ways that previous generations of scientific misconduct could not match. That is a direct attack on the verifiability foundations of reproducibility.

James: Pellegrina and Helmy in Frontiers in Artificial Intelligence reviewed AI tools now deployed to catch exactly that — fake references, image duplication, statistical inconsistencies, undisclosed AI text. These tools operate at a scale humans cannot match, and they are catching real errors in real submissions. The same technology that produces hallucinated citations is being used to detect them.

Lisa: That is an arms race, and the question is who is winning right now. Empirically, Kapoor and Narayanan, McDermott, and Day all document that AI-induced reproducibility problems are widespread today. Pellegrina and Helmy's tools are early-stage, with the authors acknowledging accuracy and adversarial-robustness limits. The harms are realized at scale; the fixes are not yet.

James: Then the honest answer is — currently worsening, plausibly fixable. The "yes" answer is right for the present moment. But it is contingent on adoption. If journals enforce Heil-style standards, deploy AI-integrity tools at the editorial layer, and use replication-prediction models in funding decisions, the trajectory could flip.

Lisa: Agreed. The verdict is yes for now, with a clear path to no — but the path requires deliberate adoption that the field has not yet committed to. The crisis is being worsened in the absence of that commitment.

Frequently asked questions

How does machine learning cause reproducibility problems in science?

A major issue is data leakage, such as train/test contamination or temporal leakage, which makes models look more accurate than they are. One review found this problem in 294 papers across 17 fields, with results failing on independent data.

Does ChatGPT make up fake citations?

A study that asked ChatGPT for references on geography topics found that most of the citations it produced were fabricated. Many had realistic author names, journals and DOIs that did not match any real publication.

Can AI help fix the reproducibility crisis?

Research shows AI can predict whether findings will replicate with accuracy comparable to expert prediction markets, at lower cost. AI tools are also being used to detect image duplication, statistical inconsistencies, invalid references and AI-generated text in manuscripts.

Are there standards for making AI research reproducible?

Researchers have proposed bronze, silver and gold reproducibility standards for machine learning in the life sciences. These rely on existing tools such as version-controlled code, containerized environments, deposited data and seeded random states, but they have not yet been widely adopted.

Related questions