Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Information

Summary

Argues standard majority voting treats all answers equally and so ignores latent heterogeneity and correlation across models, and designs two training-free aggregators, Optimal Weight and Inverse Surprising Popularity, using first- and second-order information. Theoretical analysis shows these provably mitigate majority voting's limitations under mild assumptions, validated on synthetic data, UltraFeedback, MMLU and a real-world healthcare setting. Gives the paper a principled, citable alternative to naive pooling and a formal handle on why correlation-aware weighting beats vote counting.

Contribution

Queued — not yet read.

Method

Queued — not yet read.

Relevance to RISE

Surfaced by the 2026-08-29 literature sweep as a high-priority candidate for the Architecture as Epistemology (ISR) paper (angle: aggregating LLM judgments). Verified against: Opened https://arxiv.org/abs/2510.01499 (arXiv abstract page, v2 19 May 2026; journal-ref: accepted ICML 2026)

Critique / open questions

Queued — not yet assessed.

Key quotes

With the rapid progress of multi-agent large language model (LLM) reasoning, how to effectively aggregate answers from multiple LLMs has emerged as a fundamental challenge. (abstract)

Standard majority voting treats all answers equally, failing to consider latent heterogeneity and correlation across models. (abstract)