HalluPeer: A Taxonomy-driven Benchmark for Detecting Hallucinations in Scientific Peer Reviews

Summary

Builds the first peer-review-specific hallucination benchmark: aligned triples of paper content, human-written reviews and hallucination-injected reviews, annotated for detection, classification and localization, with an induced review-specific hallucination taxonomy. Experiments over 12K papers and 38K reviews show existing hallucination detectors cannot separate fabricated claims from legitimate harsh critique. Crucially, the HalluPeer patterns are shown to occur in authentic reviews too, which makes this directly usable evidence about the reliability of LLM review assistants.

Contribution

Queued — not yet read.

Method

Queued — not yet read.

Relevance to RISE

Surfaced by the 2026-09-07 literature sweep as a high-priority candidate for the RISE knowledge base (angle: AI in peer review). Verified against: Opened https://arxiv.org/abs/2609.03580 on 2026-09-07; verbatim abstract read (12K papers, 38K reviews, injected-hallucination triples), five authors, submitted 3 Sep 2026, comment confirms acceptance to EMNLP Findings 2026. Not in dedupe_titles.txt.

Critique / open questions

Queued — not yet assessed.

Key quotes

The growing scale of academic peer review has motivated the use of Large Language Models (LLMs) as review assistants, yet LLMs can generate fluent but unsupported claims that undermine review reliability. (abstract)

Existing hallucination benchmarks are not designed for peer review, where verification requires grounding claims in long, technical papers. (abstract)