Model Retirement Creates Reproducibility Risk in Biomedical AI Publications

Summary

Uses an extraction agent over 61,077 PubMed abstracts (2022 to March 2026) to identify which LLM each biomedical study used, then joins model lifecycle data for the 50 most-used models. 77.7% of 8,931 paper-model mentions cite a closed-weight commercial model and 42% involve a model already retired at publication or retiring within two years, median publication-to-retirement 538 days. Quantifies a structural non-reproducibility risk that any replication agent or reproducibility infrastructure has to contend with, and argues model deprecation is a reporting and preservation obligation.

Contribution

Queued — not yet read.

Method

Queued — not yet read.

Relevance to RISE

Surfaced by the 2026-09-07 literature sweep as a high-priority candidate for the RISE knowledge base (angle: replication and reproducibility). Verified against: Opened https://arxiv.org/abs/2609.04699, read full abstract, 10 authors, 2026-09-04, cs.AI. Dedupe: no 'model retirement' match.

Critique / open questions

Queued — not yet assessed.

Key quotes

Background. (abstract)

Large language models (LLMs) are being adopted in biomedical research at a rapid and accelerating pace, yet commercial services that host many widely used models operate under deprecation schedules that can complicate scientific reproducibility. (abstract)