Inferred Generative-Process Diversity Predicts Correlated Failure Across Language Models

Summary

Distinguishes semantic diversity of observed outputs from generative-process diversity - differences between the processes capable of producing those outputs - and measures the latter using Normalised Compression Distance between raw model outputs residualised against a permutation control, across 38 language models. The measure recovers population structure that semantic similarity misses and predicts chance-corrected correlated failure across ten disjoint benchmark families, with a cross-benchmark partial rank association of -0.216 (95% interval -0.309 to -0.122) that is negative on all ten. This supplies the ISR paper with a defensible operationalization of the diversity a polyarchy or committee actually requires, plus a warning that embedding-similarity novelty and diversity metrics measure the wrong construct.

Contribution

Queued — not yet read.

Method

Queued — not yet read.

Relevance to RISE

Surfaced by the 2026-09-07 literature sweep as a high-priority candidate for the Architecture as Epistemology (ISR) paper (angle: correlated errors / homogeneity across LLM agents; epistemic quality of LLM outputs; MAS failure and cost analyses). Verified against: Verified via arXiv API metadata record on 2026-09-07: full abstract, 2 authors, v1, published 2026-09-03, primary cs.LG (also cs.MA), 31 pages.

Critique / open questions

Queued — not yet assessed.

Key quotes

Diversity is a widely observed factor in the resilient function of collective systems, yet the type of diversity that matters depends on the properties and failure modes of the system. (abstract)

This distinction is important for systems composed of multiple language models. (abstract)