ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Research

Summary

Benchmark of 40 end-to-end research tasks across 10 scientific domains, each grounded in a real published paper whose text is hidden at evaluation time, with expert-curated multimodal rubrics that score target-paper-level re-discovery. Seven auto-research agents and seventeen LLMs are evaluated: the best agent (Claude Code) averages 21.5 and the LLM frontier mean is 26.5, with failures concentrated in experimental-protocol mismatch, evidence mismatch and missing scientific core. Distinct from the AutoResearchClaw system paper (2605.20025) despite the similar name: this is an evaluation harness, not a system.

Contribution

Queued — not yet read.

Method

Queued — not yet read.

Relevance to RISE

Surfaced by the 2026-08-29 literature sweep as a high-priority candidate for the RISE knowledge base (angle: research-agent benchmarks). Verified against: arXiv abstract page opened 2026-08-29 (title, ~50 authors, submitted 28 May 2026, full abstract read; no reference to AutoResearchClaw / 2605.20025 in the abstract)

Critique / open questions

Queued — not yet assessed.

Key quotes

AI coding agents are increasingly used for scientific work, but their end-to-end autonomous research capability remains difficult to verify. (abstract)

We present ResearchClawBench, a benchmark for evaluating autonomous scientific research across 40 tasks from 10 scientific domains. (abstract)