Seeing Before Agreeing: Aligning Multi-Agent Consensus with Visual Evidence
About
Vision-language models (VLMs) have achieved strong performance on visual question answering (VQA). To mitigate individual hallucinations and blind spots, aggregating diverse perspectives via multi-agent collaboration has emerged as a promising paradigm. While this approach has shown great success in textual QA, its potential in the multimodal domain remains under-explored. Existing multi-agent VQA methods predominantly adapt text-centric protocols, focusing on textual discussions while ignoring the alignment of visual information. In this work, we reveal a key insight: answer-level agreement is insufficient for reliable multi-agent VQA; \textit{aligned visual evidence} -- shared support from the image regions agents rely on -- is essential for trustworthy consensus. To leverage this insight, we propose EAGLE (\textbf{E}vidence-\textbf{A}ligned \textbf{G}rounded mu\textbf{L}ti-agent r\textbf{E}asoning), a training-free evidence-centered framework for coordinating multiple VLM agents. EAGLE explicitly exposes each agent's grounding regions as visual evidence, enables mutual verification over the evidence, and uses evidence consistency to guide final decision-making. Experiments on six VQA benchmarks show that EAGLE achieves best average performance across domains while remaining lightweight, interpretable, and practical for deployment.
Key insights
What problem does EAGLE solve in multi-agent VQA?
Multi-agent VQA systems often agree on answers textually without checking whether agents rely on the same image regions. EAGLE exposes each agent's grounding boxes and visual claims, then uses evidence consistency to guide consensus.
How does EAGLE differ from prior multi-agent baselines?
Unlike text-centric debate or tool-based DART, EAGLE compares explicit grounding regions across agents. On six VQA benchmarks it averages 72.69 accuracy, beating DART by 2.54 points while using 23.2% fewer VLM calls.
Where could evidence-aligned consensus be useful?
The paper frames VQA reliability as important for medical, financial, and legal document analysis, but evaluates only benchmark prototypes. It states EAGLE is practical for deployment without documenting a real deployment.
What limitations do the authors acknowledge?
EAGLE is evaluated only on English static-image VQA benchmarks. The authors note extending to multilingual VQA, video, embodied perception, and richer evidence representations as future work.