Learning to Evaluate Before Improving: Automatic Rubric Induction for Automatic Research Agents

Summary

Addresses the underspecification problem in open-ended research tasks: AutoSciRub induces a task-specific executable rubric before execution, decomposing the instruction into atomic scientific goals grounded in literature and task-visible data, then uses criterion-level verification to drive revision of the report and its artifacts. Gains of about 2-3 points on ResearchClawBench across three backbones and three harnesses, and 16.8 points on a 20-task AstaBench E2E Discovery subset. Useful as the constructive counterpart to the audit/benchmark papers, since it makes evidential requirements explicit rather than post hoc.

Contribution

Queued — not yet read.

Method

Queued — not yet read.

Relevance to RISE

Surfaced by the 2026-09-07 literature sweep as a medium-priority candidate for the RISE knowledge base (angle: autonomous AI scientists and their evaluation). Verified against: Opened https://arxiv.org/abs/2608.31076 on 2026-09-07; verbatim abstract read (AutoSciRub, ResearchClawBench +2.08/+2.95, AstaBench +16.8), nine authors, submitted 31 Aug 2026. Not in dedupe_titles.txt.

Critique / open questions

Queued — not yet assessed.

Key quotes

Autonomous scientific research agents are increasingly applied to end-to-end scientific workflows, including literature review, data analysis, experimentation, and report generation. (abstract)

However, open-ended research tasks often do not clearly specify the analyses, methods, and success criteria required to complete the task. (abstract)