LLM hallucinations in the wild: Large-scale evidence from non-existent citations

Summary

Audits 111 million references across 2.5 million papers on arXiv, bioRxiv, SSRN and PubMed Central and estimates ~146,932 hallucinated citations in 2025 alone, concentrated in papers with AI-writing indicators and among early-career researchers. Adds an inequality finding that false attributions disproportionately accrue to prominent and male scholars, amplifying existing recognition disparities, and argues current moderation is inadequate. Written by a strong social-science/scientometrics team (Stuart, De Vaan, Ginsparg, Yin) so it travels well into management venues.

Contribution

Queued — not yet read.

Method

Queued — not yet read.

Relevance to RISE

Surfaced by the 2026-08-29 literature sweep as a high-priority candidate for the RISE knowledge base (angle: research productivity of AI). Verified against: arXiv abstract page opened 2026-08-29 (title, 6 authors, submitted 8 May 2026, full abstract read)

Critique / open questions

Queued — not yet assessed.

Key quotes

Large language models (LLMs) are known to generate plausible but false information across a wide range of contexts, yet the real-world magnitude and consequences of this hallucination problem remain poorly understood. (abstract)

Here we leverage a uniquely verifiable object - scientific citations - to audit 111 million references across 2.5 million papers in arXiv, bioRxiv, SSRN, and PubMed Central. (abstract)