Knowledge Reasoning on MMLU (Accuracy)
92.16MMLU Knowledge Reasoning AccuracySIGMA
Evaluation Results
| Method | Links | |
|---|---|---|
| SIGMAMulti-agent collaboration (Mul.)=full, Inter-agent relations (Rel.)=full, Conflicting signals (Conf.)=full, Backbone=DeepSeek-V3.22026.05 | 92.16 | |
| GoAMulti-agent collaboration (Mul.)=full, Inter-agent relations (Rel.)=full, Conflicting signals (Conf.)=partial, Backbone=DeepSeek-V3.22026.05 | 91.53 | |
| G-DesignerMulti-agent collaboration (Mul.)=full, Inter-agent relations (Rel.)=full, Conflicting signals (Conf.)=no, Backbone=DeepSeek-V3.22026.05 | 90.85 | |
| GPTSwarmMulti-agent collaboration (Mul.)=full, Inter-agent relations (Rel.)=full, Conflicting signals (Conf.)=no, Backbone=DeepSeek-V3.22026.05 | 90.03 | |
| ST-EVOBackbone=gpt-oss-120b2026.02 | 89.85 | |
| MoAMulti-agent collaboration (Mul.)=full, Inter-agent relations (Rel.)=no, Conflicting signals (Conf.)=no, Backbone=DeepSeek-V3.22026.05 | 89.77 | |
| ComplexCoTMulti-agent collaboration (Mul.)=no, Inter-agent relations (Rel.)=no, Conflicting signals (Conf.)=no, Backbone=DeepSeek-V3.22026.05 | 89.41 | |
| CoTMulti-agent collaboration (Mul.)=no, Inter-agent relations (Rel.)=no, Conflicting signals (Conf.)=no, Backbone=DeepSeek-V3.22026.05 | 88.74 | |
| VanillaMulti-agent collaboration (Mul.)=no, Inter-agent relations (Rel.)=no, Conflicting signals (Conf.)=no, Backbone=DeepSeek-V3.22026.05 | 87.46 | |
| Qwen3Params=32B2026.04 | 87.2 | |
| I-DLMParams=32B, Decoding=ISD (N=4, sampling)2026.04 | 86.8 | |
| ARG-DesignerBackbone=gpt-oss-120b2026.02 | 85.04 | |
| SafeSieveBackbone=gpt-oss-120b2026.02 | 84.65 | |
| G-DesignerBackbone=gpt-oss-120b2026.02 | 84.63 | |
| STEERBackbone=gpt-oss-120b2026.02 | 84.4 | |
| SpecReasonBackbone=gpt-oss-120b2026.02 | 84.2 | |
| LangGraphBackbone=gpt-oss-120b2026.02 | 84.1 | |
| GPTSwarmBackbone=gpt-oss-120b2026.02 | 83.8 | |
| Qwen3Params=8B2026.04 | 83.5 | |
| LLM-DebateBackbone=gpt-oss-120b2026.02 | 83.28 | |
| AutoGenBackbone=gpt-oss-120b2026.02 | 83.25 | |
| AFlowBackbone=gpt-oss-120b2026.02 | 83.22 | |
| Random GraphBackbone=gpt-oss-120b2026.02 | 83.2 | |
| SCBackbone=gpt-oss-120b2026.02 | 82.95 | |
| SDARParams=30B2026.04 | 82.8 | |
| Complete GraphBackbone=gpt-oss-120b2026.02 | 82.75 | |
| ChainBackbone=gpt-oss-120b2026.02 | 82.4 | |
| I-DLMParams=8B, Decoding=ISD (N=4, sampling)2026.04 | 82.4 | |
| CoTBackbone=gpt-oss-120b2026.02 | 81.85 | |
| TreeBackbone=gpt-oss-120b2026.02 | 81.55 | |
| VanillaBackbone=gpt-oss-120b2026.02 | 80.47 | |
| StarBackbone=gpt-oss-120b2026.02 | 79.64 | |
| SDARParams=8B2026.04 | 78.6 | |
| POESOptimizer=OPRO, Model=Qwen-2.5-7B-Instruct2026.04 | 77.9 | |
| LLaDA-2.1-miniParams=16B2026.04 | 74.5 | |
| AnchorOptimizer=OPRO, Model=Qwen-2.5-7B-Instruct2026.04 | 71 | |
| IPOMPOptimizer=OPRO, Model=Qwen-2.5-7B-Instruct2026.04 | 70.1 | |
| SESSOptimizer=OPRO, Model=Qwen-2.5-7B-Instruct2026.04 | 69.8 | |
| RandomOptimizer=OPRO, Model=Qwen-2.5-7B-Instruct2026.04 | 68.8 | |
| PredictionOptimizer=OPRO, Model=Qwen-2.5-7B-Instruct2026.04 | 68.7 | |
| POESOptimizer=EvoPrompt-DE, Model=Qwen-2.5-7B-Instruct2026.04 | 66.6 | |
| POESOptimizer=EvoPrompt-GA, Model=Qwen-2.5-7B-Instruct2026.04 | 66.4 | |
| Qwen2.5-VL 3BText Source=Pure-Text, Zero-shot setup=true2025.10 | 61.91 | |
| POESOptimizer=EvoPrompt-DE, Model=Llama-3.1-8B-Instruct2026.04 | 59.5 | |
| PredictionOptimizer=EvoPrompt-GA, Model=Qwen-2.5-7B-Instruct2026.04 | 58.5 | |
| POESOptimizer=EvoPrompt-GA, Model=Llama-3.1-8B-Instruct2026.04 | 57.1 | |
| PredictionOptimizer=EvoPrompt-DE, Model=Qwen-2.5-7B-Instruct2026.04 | 56.7 | |
| POESOptimizer=OPRO, Model=Llama-3.1-8B-Instruct2026.04 | 56.5 | |
| SESSOptimizer=EvoPrompt-DE, Model=Qwen-2.5-7B-Instruct2026.04 | 55.8 | |
| RandomOptimizer=EvoPrompt-GA, Model=Qwen-2.5-7B-Instruct2026.04 | 55.5 | |
| RandomOptimizer=EvoPrompt-DE, Model=Qwen-2.5-7B-Instruct2026.04 | 55.2 | |
| IPOMPOptimizer=EvoPrompt-DE, Model=Qwen-2.5-7B-Instruct2026.04 | 54.9 | |
| AnchorOptimizer=EvoPrompt-DE, Model=Qwen-2.5-7B-Instruct2026.04 | 54.8 | |
| AnchorOptimizer=EvoPrompt-GA, Model=Qwen-2.5-7B-Instruct2026.04 | 54.6 | |
| IPOMPOptimizer=EvoPrompt-GA, Model=Qwen-2.5-7B-Instruct2026.04 | 53.9 | |
| SESSOptimizer=EvoPrompt-GA, Model=Qwen-2.5-7B-Instruct2026.04 | 53.7 | |
| SEETOKText Source=Visual-Text, Zero-shot setup=true2025.10 | 52.52 | |
| RandomOptimizer=EvoPrompt-DE, Model=Llama-3.1-8B-Instruct2026.04 | 49.9 | |
| PredictionOptimizer=EvoPrompt-DE, Model=Llama-3.1-8B-Instruct2026.04 | 49.1 | |
| AnchorOptimizer=EvoPrompt-GA, Model=Llama-3.1-8B-Instruct2026.04 | 48.8 | |
| IPOMPOptimizer=EvoPrompt-GA, Model=Llama-3.1-8B-Instruct2026.04 | 48.5 | |
| SESSOptimizer=EvoPrompt-GA, Model=Llama-3.1-8B-Instruct2026.04 | 48.1 | |
| AnchorOptimizer=EvoPrompt-DE, Model=Llama-3.1-8B-Instruct2026.04 | 48.1 | |
| SESSOptimizer=EvoPrompt-DE, Model=Llama-3.1-8B-Instruct2026.04 | 47 | |
| PredictionOptimizer=EvoPrompt-GA, Model=Llama-3.1-8B-Instruct2026.04 | 46.5 | |
| IPOMPOptimizer=EvoPrompt-DE, Model=Llama-3.1-8B-Instruct2026.04 | 45.9 | |
| IPOMPOptimizer=OPRO, Model=Llama-3.1-8B-Instruct2026.04 | 45.5 | |
| RandomOptimizer=EvoPrompt-GA, Model=Llama-3.1-8B-Instruct2026.04 | 44.6 | |
| AnchorOptimizer=OPRO, Model=Llama-3.1-8B-Instruct2026.04 | 44.2 | |
| RandomOptimizer=OPRO, Model=Llama-3.1-8B-Instruct2026.04 | 44 | |
| PredictionOptimizer=OPRO, Model=Llama-3.1-8B-Instruct2026.04 | 43.7 | |
| SESSOptimizer=OPRO, Model=Llama-3.1-8B-Instruct2026.04 | 43.5 | |
| Qwen2.5-VL 3BText Source=Visual-Text, Zero-shot setup=true2025.10 | 32.31 |