BixBench3: Benchmarking AI agents on research-study-scale computational biology tasks

Summary

FutureHouse-affiliated benchmark measuring whether agents can take raw biological data through to scientific results at the scale of a complete research study: 20 tasks, 138 artifacts, frontier models scoring 0.00-0.48. Performance degrades with raw dataset size (0.36 under 100 GB vs 0.10 over 100 GB) and with the number of sequential analysis steps, and an average task consumed 6.8 hours, 102M tokens and $43. Rare in reporting the compute-and-cost envelope alongside capability, which is what makes agentic research pipelines comparable.

Contribution

Queued — not yet read.

Method

Queued — not yet read.

Relevance to RISE

Surfaced by the 2026-09-07 literature sweep as a high-priority candidate for the RISE knowledge base (angle: research-agent benchmarks). Verified against: Opened https://arxiv.org/abs/2608.25286, read abstract, 8 authors, 2026-08-26, cs.AI/q-bio.QM. Dedupe: no 'bixbench' match.

Critique / open questions

Queued — not yet assessed.

Key quotes

Artificial intelligence (AI) promises to accelerate biological research by automating computational analyses. (abstract)

Yet the ability of AI agents to carry out computational biology at the scale of complete research studies has not been systematically evaluated. (abstract)