Knowledge Reasoning on MMLU-Pro
91.43AccuracySIGMA
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SIGMAMulti-agent collaboration (Mul.)=full, Inter-agent relations (Rel.)=full, Conflicting signals (Conf.)=full, Backbone=DeepSeek-V3.22026.05 | 91.43 | 93.91 | — | — | — | — | |
| G-DesignerMulti-agent collaboration (Mul.)=full, Inter-agent relations (Rel.)=full, Conflicting signals (Conf.)=no, Backbone=DeepSeek-V3.22026.05 | 90.54 | 93.22 | — | — | — | — | |
| GoAMulti-agent collaboration (Mul.)=full, Inter-agent relations (Rel.)=full, Conflicting signals (Conf.)=partial, Backbone=DeepSeek-V3.22026.05 | 90.24 | 92.86 | — | — | — | — | |
| GPTSwarmMulti-agent collaboration (Mul.)=full, Inter-agent relations (Rel.)=full, Conflicting signals (Conf.)=no, Backbone=DeepSeek-V3.22026.05 | 90.12 | 92.39 | — | — | — | — | |
| ComplexCoTMulti-agent collaboration (Mul.)=no, Inter-agent relations (Rel.)=no, Conflicting signals (Conf.)=no, Backbone=DeepSeek-V3.22026.05 | 86.45 | 91.01 | — | — | — | — | |
| CoTMulti-agent collaboration (Mul.)=no, Inter-agent relations (Rel.)=no, Conflicting signals (Conf.)=no, Backbone=DeepSeek-V3.22026.05 | 85.72 | 90.68 | — | — | — | — | |
| In-placeModel=Gemma-3-27B, Number of turns=42025.10 | 83.4 | — | — | — | — | — | |
| In-placeModel=Gemma-3-27B, Number of turns=32025.10 | 81.7 | — | — | — | — | — | |
| MemoryModel=Gemma-3-27B, Number of turns=42025.10 | 81.4 | — | — | — | — | — | |
| In-placeModel=Llama-3.1-70B, Number of turns=42025.10 | 81.3 | — | — | — | — | — | |
| MoAMulti-agent collaboration (Mul.)=full, Inter-agent relations (Rel.)=no, Conflicting signals (Conf.)=no, Backbone=DeepSeek-V3.22026.05 | 80.76 | 89.83 | — | — | — | — | |
| Multi-turnModel=Gemma-3-27B, Number of turns=42025.10 | 80.5 | — | — | — | — | — | |
| MemoryModel=Gemma-3-27B, Number of turns=32025.10 | 80.3 | — | — | — | — | — | |
| MemoryModel=Llama-3.1-70B, Number of turns=42025.10 | 80.2 | — | — | — | — | — | |
| Qwen3Params=32B2026.04 | 80.1 | — | — | — | — | — | |
| Qwen3-32BSetting=Steer2026.06 | 80.03 | — | — | — | — | — | |
| Qwen3-32BSetting=Recur2026.06 | 79.72 | — | — | — | — | 2.42 | |
| I-DLMParams=32B, Decoding=ISD (N=4, sampling)2026.04 | 79.7 | — | — | — | — | — | |
| LLaMA-70BSetting=Steer2026.06 | 79.4 | — | — | — | — | — | |
| Multi-turnModel=Gemma-3-27B, Number of turns=32025.10 | 79.3 | — | — | — | — | — | |
| In-placeModel=Gemma-3-27B, Number of turns=22025.10 | 79.3 | — | — | — | — | — | |
| LLaMA-70BSetting=Recur2026.06 | 79.15 | — | — | — | — | 3.27 | |
| In-placeModel=Llama-3.1-70B, Number of turns=32025.10 | 79.1 | — | — | — | — | — | |
| MemoryModel=Gemma-3-27B, Number of turns=22025.10 | 78.7 | — | — | — | — | — | |
| MemoryModel=Llama-3.1-70B, Number of turns=32025.10 | 78.5 | — | — | — | — | — | |
| Multi-turnModel=Llama-3.1-70B, Number of turns=42025.10 | 77.9 | — | — | — | — | — | |
| Qwen3-32BSetting=Base2026.06 | 77.83 | — | — | — | — | — | |
| Qwen3-32BSetting=Base2026.06 | 77.83 | — | — | — | — | — | |
| Qwen3-14BSetting=Recur2026.06 | 77.79 | — | — | — | — | 5.77 | |
| In-placeModel=Llama-3.1-70B, Number of turns=22025.10 | 77.7 | — | — | — | — | — | |
| Qwen2.5-32B-Instruct + Bootcamp-SFT-RLModel=Qwen2.5-32B-Instruct, Training Stage=Bootcamp-SFT-RL2025.08 | 77.1 | — | — | — | — | — | |
| Multi-turnModel=Llama-3.1-70B, Number of turns=32025.10 | 77.1 | — | — | — | — | — | |
| LLaMA-70BSetting=Base2026.06 | 76.64 | — | — | — | — | — | |
| LLaMA-70BSetting=Base2026.06 | 76.64 | — | — | — | — | — | |
| LLaDA-2.1-flashParams=100B2026.04 | 76.6 | — | — | — | — | — | |
| Qwen3-14BSetting=Steer2026.06 | 76.57 | — | — | — | — | — | |
| MemoryModel=Llama-3.1-70B, Number of turns=22025.10 | 76.5 | — | — | — | — | — | |
| Multi-turnModel=Gemma-3-27B, Number of turns=22025.10 | 76.3 | — | — | — | — | — | |
| SCOPE-RLBase Model=Qwen3-4B2025.10 | 76.1 | — | — | — | — | — | |
| Qwen2.5-32B-Instruct + Bootcamp-SFTModel=Qwen2.5-32B-Instruct, Training Stage=Bootcamp-SFT2025.08 | 76.1 | — | — | — | — | — | |
| VanillaMulti-agent collaboration (Mul.)=no, Inter-agent relations (Rel.)=no, Conflicting signals (Conf.)=no, Backbone=DeepSeek-V3.22026.05 | 75.71 | 87.69 | — | — | — | — | |
| KL-covBase Model=Qwen3-4B2025.10 | 75.5 | — | — | — | — | — | |
| Multi-turnModel=Llama-3.1-70B, Number of turns=22025.10 | 75.3 | — | — | — | — | — | |
| Qwen3-8BSetting=Recur2026.06 | 75.24 | — | — | — | — | 5.94 | |
| Qwen3Params=8B2026.04 | 75.1 | — | — | — | — | — | |
| CISPOBase Model=Qwen3-4B2025.10 | 75.1 | — | — | — | — | — | |
| DAPOBase Model=Qwen3-4B2025.10 | 75 | — | — | — | — | — | |
| LLaDA-2.0-flashParams=100B2026.04 | 74.8 | — | — | — | — | — | |
| Entropy-AdvBase Model=Qwen3-4B2025.10 | 74.8 | — | — | — | — | — | |
| MemoryModel=Gemma-3-27B, Number of turns=12025.10 | 74.6 | — | — | — | — | — | |
| In-placeModel=Gemma-3-27B, Number of turns=12025.10 | 74.4 | — | — | — | — | — | |
| Qwen3-8BSetting=Steer2026.06 | 74.21 | — | — | — | — | — | |
| GRPOBase Model=Qwen3-4B2025.10 | 74.1 | — | — | — | — | — | |
| Entropy-RegBase Model=Qwen3-4B2025.10 | 73.7 | — | — | — | — | — | |
| Qwen3-14BSetting=Base2026.06 | 73.56 | — | — | — | — | — | |
| Qwen3-14BSetting=Base2026.06 | 73.56 | — | — | — | — | — | |
| Multi-turnModel=Gemma-3-27B, Number of turns=12025.10 | 73.4 | — | — | — | — | — | |
| I-DLMParams=8B, Decoding=ISD (N=4, sampling)2026.04 | 73.1 | — | — | — | — | — | |
| DS-R1-Distilled-Qwen-32BModel=DS-R1-Distilled-Qwen-32B2025.08 | 73 | — | — | — | — | — | |
| BaseBase Model=Qwen3-4B2025.10 | 72.6 | — | — | — | — | — | |
| Qwen3-4BSetting=Recur2026.06 | 71.66 | — | — | — | — | 5.5 | |
| Qwen3-4BSetting=Steer2026.06 | 71.15 | — | — | — | — | — | |
| Qwen3-8BSetting=Base2026.06 | 71.11 | — | — | — | — | — | |
| Qwen3-8BSetting=Base2026.06 | 71.11 | — | — | — | — | — | |
| DS-R1-Distilled-Qwen-32B + Bootcamp-RLModel=DS-R1-Distilled-Qwen-32B, Training Stage=Bootcamp-RL2025.08 | 70.6 | — | — | — | — | — | |
| In-placeModel=Llama-3.1-70B, Number of turns=12025.10 | 70.6 | — | — | — | — | — | |
| Multi-turnModel=Llama-3.1-70B, Number of turns=12025.10 | 70.2 | — | — | — | — | — | |
| Qwen2.5-32B-InstructModel=Qwen2.5-32B-Instruct2025.08 | 69.1 | — | — | — | — | — | |
| MemoryModel=Llama-3.1-70B, Number of turns=12025.10 | 68.8 | — | — | — | — | — | |
| Qwen2.5-32B-Instruct + Bootcamp-RLModel=Qwen2.5-32B-Instruct, Training Stage=Bootcamp-RL2025.08 | 68.3 | — | — | — | — | — | |
| Qwen3-4BSetting=Base2026.06 | 67.92 | — | — | — | — | — | |
| Qwen3-4BSetting=Base2026.06 | 67.92 | — | — | — | — | — | |
| LLaDA-2.1-miniParams=16B2026.04 | 64.8 | — | — | — | — | — | |
| SDPOBase Model=Qwen3-8B, Evaluation Protocol=Chain-of-Thought2026.02 | 63.3 | — | — | — | — | — | |
| Qwen3-8BModel Size=8B, Evaluation Protocol=Chain-of-Thought2026.02 | 62.5 | — | — | — | — | — | |
| SDARParams=30B2026.04 | 61.5 | — | — | — | — | — | |
| Multi-turnModel=Gemma-3-27B, Number of turns=02025.10 | 61.3 | — | — | — | — | — | |
| MemoryModel=Gemma-3-27B, Number of turns=02025.10 | 61.3 | — | — | — | — | — | |
| In-placeModel=Gemma-3-27B, Number of turns=02025.10 | 61.3 | — | — | — | — | — | |
| LOVER (sum)LLM=qwen2.5-7b-instruct2026.05 | 61.24 | — | — | — | — | — | |
| SupervisedLLM=qwen2.5-7b-instruct2026.05 | 61.13 | — | — | — | — | — | |
| SCOPE-RLBase Model=Qwen2.5-7B2025.10 | 60.7 | — | — | — | — | — | |
| LLaMA-8BSetting=Steer2026.06 | 60.67 | — | — | — | — | — | |
| LLaMA-8BSetting=Recur2026.06 | 60.47 | — | — | — | — | 2.03 | |
| DAPOBase Model=Qwen2.5-7B2025.10 | 59.8 | — | — | — | — | — | |
| LOVER (max)LLM=qwen2.5-7b-instruct2026.05 | 59.28 | — | — | — | — | — | |
| LLaMA-8BSetting=Base2026.06 | 59.27 | — | — | — | — | — | |
| LLaMA-8BSetting=Base2026.06 | 59.27 | — | — | — | — | — | |
| KL-covBase Model=Qwen2.5-7B2025.10 | 59.1 | — | — | — | — | — | |
| CoT-Decoding (sum)LLM=qwen2.5-7b-instruct2026.05 | 58.81 | — | — | — | — | — | |
| Qwen3-1.7BSetting=Recur2026.06 | 58.63 | — | — | — | — | 4.82 | |
| Qwen3-1.7BSetting=Steer2026.06 | 58.49 | — | — | — | — | — | |
| GRPOBase Model=Qwen2.5-7B2025.10 | 58.2 | — | — | — | — | — | |
| Entropy-RegBase Model=Qwen2.5-7B2025.10 | 58.2 | — | — | — | — | — | |
| Qwen3-4BModel Size=4B, Evaluation Protocol=Chain-of-Thought2026.02 | 58.1 | — | — | — | — | — | |
| SDPOBase Model=Qwen3-4B, Evaluation Protocol=Chain-of-Thought2026.02 | 58 | — | — | — | — | — | |
| CISPOBase Model=Qwen2.5-7B2025.10 | 57.8 | — | — | — | — | — | |
| Entropy-AdvBase Model=Qwen2.5-7B2025.10 | 57.7 | — | — | — | — | — | |
| Majority VotingLLM=qwen2.5-7b-instruct2026.05 | 57 | — | — | — | — | — | |
| SDARParams=8B2026.04 | 56.9 | — | — | — | — | — |