Question Answering on MMLU Pro.Med.
95.59AccuracyMA-PoP
Evaluation Results
| Method | Links | |
|---|---|---|
| MA-PoPAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, Multi-Agent Strategy=Multi-Agent2026.05 | 95.59 | |
| Free MADAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, T (Number of agents/steps)=3, Multi-Agent Strategy=Multi-Agent2026.05 | 94.49 | |
| Free MADAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, T (Number of agents/steps)=5, Multi-Agent Strategy=Multi-Agent2026.05 | 94.49 | |
| Decentr. MADAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, T (Number of agents/steps)=2, Multi-Agent Strategy=Multi-Agent2026.05 | 93.75 | |
| Decentr. MADAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, T (Number of agents/steps)=5, Multi-Agent Strategy=Multi-Agent2026.05 | 93.75 | |
| Sparse MADAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, T (Number of agents/steps)=2, Multi-Agent Strategy=Multi-Agent2026.05 | 93.75 | |
| Free MADAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, T (Number of agents/steps)=2, Multi-Agent Strategy=Multi-Agent2026.05 | 93.75 | |
| Self-ConsistencyAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, Multi-Agent Strategy=Multi-Agent2026.05 | 93.75 | |
| Sparse MADAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, T (Number of agents/steps)=3, Multi-Agent Strategy=Multi-Agent2026.05 | 93.64 | |
| Decentr. MADAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, T (Number of agents/steps)=3, Multi-Agent Strategy=Multi-Agent2026.05 | 93.38 | |
| Sparse MADAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, T (Number of agents/steps)=5, Multi-Agent Strategy=Multi-Agent2026.05 | 93.38 | |
| Majority VotingAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, Multi-Agent Strategy=Multi-Agent2026.05 | 93.38 | |
| LLPAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, Multi-Agent Strategy=Multi-Agent2026.05 | 93.38 | |
| ISPAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, Multi-Agent Strategy=Multi-Agent2026.05 | 92.28 | |
| Falcon-34BMulti-Agent Strategy=Single-Agent2026.05 | 91.91 | |
| Centr. MADAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, T (Number of agents/steps)=2, Multi-Agent Strategy=Multi-Agent2026.05 | 90.44 | |
| Centr. MADAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, T (Number of agents/steps)=3, Multi-Agent Strategy=Multi-Agent2026.05 | 90.07 | |
| Centr. MADAgent Composition (Heterogeneous)=Qwen-32B, Falcon-34B, Gemma-27B, T (Number of agents/steps)=5, Multi-Agent Strategy=Multi-Agent2026.05 | 90.07 | |
| Qwen-32BMulti-Agent Strategy=Single-Agent2026.05 | 88.97 | |
| MA-PoPAggregation Method=MA-PoP, Number of Agents (M)=5, Agent Diversity=Heterogeneous2026.05 | 87.87 | |
| Free MADNumber of Agents (M)=5, Number of Debate Rounds (T)=5, Agent Diversity=Heterogeneous2026.05 | 87.5 | |
| Free MADNumber of Agents (M)=5, Number of Debate Rounds (T)=2, Agent Diversity=Heterogeneous2026.05 | 87.13 | |
| Free MADNumber of Agents (M)=5, Number of Debate Rounds (T)=3, Agent Diversity=Heterogeneous2026.05 | 87.13 | |
| Decentr. MADNumber of Agents (M)=5, Number of Debate Rounds (T)=5, Agent Diversity=Heterogeneous2026.05 | 86.76 | |
| ISPAggregation Method=ISP, Number of Agents (M)=5, Agent Diversity=Heterogeneous2026.05 | 86.4 | |
| Decentr. MADNumber of Agents (M)=5, Number of Debate Rounds (T)=2, Agent Diversity=Heterogeneous2026.05 | 86.03 | |
| Decentr. MADNumber of Agents (M)=5, Number of Debate Rounds (T)=3, Agent Diversity=Heterogeneous2026.05 | 86.03 | |
| Sparse MADNumber of Agents (M)=5, Number of Debate Rounds (T)=2, Agent Diversity=Heterogeneous2026.05 | 86.03 | |
| Self-ConsistencyAggregation Method=Self-Consistency, Number of Agents (M)=5, Agent Diversity=Heterogeneous2026.05 | 86.03 | |
| LLPAggregation Method=LLP, Number of Agents (M)=5, Agent Diversity=Heterogeneous2026.05 | 85.67 | |
| Sparse MADNumber of Agents (M)=5, Number of Debate Rounds (T)=3, Agent Diversity=Heterogeneous2026.05 | 85.29 | |
| Sparse MADNumber of Agents (M)=5, Number of Debate Rounds (T)=5, Agent Diversity=Heterogeneous2026.05 | 85.29 | |
| Gemma-27BMulti-Agent Strategy=Single-Agent2026.05 | 84.93 | |
| Majority VotingAggregation Method=Majority Voting, Number of Agents (M)=5, Agent Diversity=Heterogeneous2026.05 | 84.93 | |
| Centr. MADNumber of Agents (M)=5, Number of Debate Rounds (T)=5, Agent Diversity=Heterogeneous2026.05 | 84.19 | |
| Centr. MADNumber of Agents (M)=5, Number of Debate Rounds (T)=2, Agent Diversity=Heterogeneous2026.05 | 83.72 | |
| Centr. MADNumber of Agents (M)=5, Number of Debate Rounds (T)=3, Agent Diversity=Heterogeneous2026.05 | 83.33 | |
| Gemma-9BNumber of Agents (M)=1, Number of Debate Rounds (T)=12026.05 | 80.15 | |
| Falcon-7BNumber of Agents (M)=1, Number of Debate Rounds (T)=12026.05 | 79.04 | |
| Qwen-7BNumber of Agents (M)=1, Number of Debate Rounds (T)=12026.05 | 78.68 | |
| Llama-8BNumber of Agents (M)=1, Number of Debate Rounds (T)=12026.05 | 74.41 | |
| Mistral-7BNumber of Agents (M)=1, Number of Debate Rounds (T)=12026.05 | 65.44 |