More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration

Summary

Builds a turn-based multi-agent environment where cooperation is costless and trivially optimal, and finds across eight widely used LLMs that capability does not predict cooperation - OpenAI o3 reaches only 17% of optimal collective performance while the weaker o3-mini reaches 50% under identical instructions to maximise group revenue. A causal decomposition that automates one side of agent communication separates cooperation failures from competence failures, showing several capable models actively withhold information despite gaining nothing from doing so; explicit protocols roughly double the performance of competence-limited models while small sharing incentives unlock cooperation-limited ones. Direct evidence that structural and protocol design, not model strength, governs collective epistemic output.

Contribution

Queued — not yet read.

Method

Queued — not yet read.

Relevance to RISE

Surfaced by the 2026-08-29 literature sweep as a medium-priority candidate for the Architecture as Epistemology (ISR) paper (angle: MAS topology / structure vs. aggregation rule; organization theory applied to AI agents; MAS failure and cost analyses). Verified against: Opened https://arxiv.org/abs/2604.07821 (arXiv abstract page, v2 4 Jun 2026; journal-ref: accepted ICML 2026 main conference)

Critique / open questions

Queued — not yet assessed.

Key quotes

Large language model (LLM) agents increasingly coordinate in multi-agent systems, yet we lack an understanding of where and why cooperation fails. (abstract)

Many real-world coordination problems are not social dilemmas: helping others -- sharing documentation, unblocking a teammate -- costs the helper almost nothing while producing substantial collective benefit. (abstract)