# When Agents Disagree: The Selection Bottleneck in Multi-Agent LLM Pipelines

Source: [When Agents Disagree: The Selection Bottleneck in Multi-Agent LLM Pipelines](<https://www.sota2.com/research/paper/69c21a880636e5bf4afc4820>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-07-23T05:00:05.031Z

Authors: Artem Maryanskyy, Dmitry Budnikov, Alibek T. Kaliyev

Published: 2026-03-20T00:50:53.000Z

## Abstract

Multi-agent LLM pipelines produce contradictory evidence on whether team diversity improves output quality: heterogeneous Mixture-of-Agents teams outperform single models, yet homogeneous Self-MoA teams consistently win under synthesis-based aggregation. We propose a resolution by identifying the selection bottleneck -- a crossover threshold in aggregation quality that determines whether diversity helps or hurts. Under this model, we obtain a closed-form crossover threshold $s^\*$ (Proposition 1) that separates the regimes where diversity helps and hurts. In a targeted experiment spanning 42 tasks across 7 categories ($N=210$), a diverse team with judge-based selection achieves a win rate of 0.810 against a single-model baseline, while a homogeneous team scores 0.512 -- near chance (Glass's $\\Delta = 2.07$). Judge-based selection outperforms MoA-style synthesis by $\\Delta\_{\\mathrm{WR}} = +0.631$ -- the synthesis approach is preferred over the baseline in zero of 42 tasks by the judge panel. A decoupled evaluation with independent judges confirms all directional findings (Spearman $\\rho = 0.90$). Exploratory evidence suggests that including a weaker model improves performance while reducing cost ($p \< 10^{-4}$, not pre-registered). Our results suggest that selector quality may be a more impactful design lever than generator diversity in single-round generate-then-select pipelines.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Multi-agent aggregation on 42-task multi-agent evaluation set](<https://www.sota2.com/research/sota/multi-agent-aggregation-on-42-task-multi-agent-evaluation-set.md>): Multi-agent aggregation; 42-task multi-agent evaluation set; BT-WR: 92.9.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2603.20324>)

[PDF](<https://arxiv.org/pdf/2603.20324.pdf>)