Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents

Summary

Systematic measurement of citation-URL reliability across 10 models and agents on DRBench (53,090 URLs) and 3 models on ExpertQA (168,021 URLs, 32 academic fields): 3-13% of citation URLs are hallucinated with no Wayback Machine record, 5-18% do not resolve at all, and deep-research agents both cite more and hallucinate at higher rates than search-augmented LLMs. Non-resolving rates vary by field from 5.4% (Business) to 11.4% (Theology). Ships urlhealth, an open-source liveness checker that measurably reduces bad citations in agentic self-correction.

Contribution

Queued — not yet read.

Method

Queued — not yet read.

Relevance to RISE

Surfaced by the 2026-08-29 literature sweep as a high-priority candidate for the RISE knowledge base (angle: research-agent benchmarks). Verified against: arXiv abstract page opened 2026-08-29 (title, 3 authors, submitted 3 Apr 2026, full abstract with statistics read)

Critique / open questions

Queued — not yet assessed.

Key quotes

Large language models and deep research agents supply citation URLs to support their claims, yet the reliability of these citations has not been systematically measured. (abstract)

We address six research questions about citation URL validity using 10 models and agents on DRBench (53,090 URLs) and 3 models on ExpertQA (168,021 URLs across 32 academic fields). (abstract)