Beyond Human-Likeness: Mapping the Scientific Critique Profiles of LLMs and Human Reviewers

Summary

Rejects human-likeness as the evaluation target for LLM reviewing and instead annotates ICLR 2025 human and LLM review points against five theory-guided frameworks (Anderson, Toulmin, Graesser, SOLO, Hattie) to characterise what critique functions each performs. Humans supply more scientific framing, higher-order weaknesses and revision guidance; LLMs supply more explanatory depth and explicit argument structuring, and expert prompting amplifies rather than corrects those LLM tendencies. A methodologically careful alternative to overlap-based LLM-reviewer benchmarks.

Contribution

Queued — not yet read.

Method

Queued — not yet read.

Relevance to RISE

Surfaced by the 2026-09-07 literature sweep as a high-priority candidate for the RISE knowledge base (angle: AI in peer review). Verified against: Opened https://arxiv.org/abs/2609.01895, read full abstract, 3 authors, 2026-09-01, cs.DL. Dedupe: no 'critique profiles' match.

Critique / open questions

Queued — not yet assessed.

Key quotes

Large language models (LLMs) are increasingly discussed as tools for peer review, but their value is often assessed through human-likeness, perceived usefulness, or textual overlap with reviewer comments. (abstract)

This study shifts attention from whether LLMs resemble human reviewers to what functions of scientific critique they perform. (abstract)