Design and Evaluation of Multi-Agent AI Oracle Systems for Prediction Market Resolution

Summary

Explicitly compares independent aggregation against deliberative consensus on 1,189 resolved prediction-market questions, holding the evidence layer constant so reasoning is isolated from retrieval quality. Independent aggregation with confidence-weighted voting is best at 83.43%, while deliberative consensus degrades to about 76%, below every single-model baseline, attributed to error propagation where confidently wrong models flip correct ones. It further reports cross-model error correlations of 0.529-0.689 and argues these place a fundamental limit below the theoretical Condorcet ceiling. The paper's independent-versus-deliberative contrast and its confidence-weighting aggregation rule, on a forecasting task, in a single study.

Contribution

Queued — not yet read.

Method

Queued — not yet read.

Relevance to RISE

Surfaced by the 2026-08-29 literature sweep as a high-priority candidate for the Architecture as Epistemology (ISR) paper (angle: aggregating LLM judgments; LLM agents in forecasting; correlated errors / homogeneity across LLM agents; MAS topology / structure vs. aggregation rule). Verified against: Opened https://arxiv.org/abs/2605.30802 (arXiv abstract page, v1 29 May 2026)

Critique / open questions

Queued — not yet assessed.

Key quotes

Prediction markets aggregate collective intelligence to forecast uncertain events, but their utility depends on reliable outcome resolution. (abstract)

Existing oracle systems tradeoff fast but brittle automation against accurate but costly human arbitration. (abstract)