Mathematical Reasoning on GSM8K (test) (Accuracy, Reward)
96.23AccuracySIGMA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SIGMAMulti-agent collaboration=full support, Inter-agent relations=full support, Handling conflicting signals=full support2026.05 | 96.23 | — | |
| G-DesignerMulti-agent collaboration=full support, Inter-agent relations=full support2026.05 | 96.1 | — | |
| PHPMulti-agent collaboration=full support, Inter-agent relations=no support, Handling conflicting signals=no support2026.05 | 95.87 | — | |
| GPTSwarmMulti-agent collaboration=full support, Inter-agent relations=full support2026.05 | 95.31 | — | |
| GoAMulti-agent collaboration=full support, Inter-agent relations=full support2026.05 | 95.25 | — | |
| AutoGenMulti-agent collaboration=full support, Inter-agent relations=full support2026.05 | 95.13 | — | |
| SCMulti-agent collaboration=no support, Inter-agent relations=no support, Handling conflicting signals=no support2026.05 | 94.76 | — | |
| ComplexCoTMulti-agent collaboration=no support, Inter-agent relations=no support, Handling conflicting signals=no support2026.05 | 94.74 | — | |
| CoTMulti-agent collaboration=no support, Inter-agent relations=no support, Handling conflicting signals=no support2026.05 | 94.47 | — | |
| Self-MoAMulti-agent collaboration=full support, Inter-agent relations=no support, Handling conflicting signals=no support2026.05 | 94.34 | — | |
| Complete GraphMulti-agent collaboration=full support, Inter-agent relations=full support, Handling conflicting signals=no support2026.05 | 94.21 | — | |
| Random GraphMulti-agent collaboration=full support, Inter-agent relations=full support, Handling conflicting signals=no support2026.05 | 94.15 | — | |
| MANGOBase LLM=GPT-4o-mini2026.05 | 94.12 | — | |
| MoAMulti-agent collaboration=full support, Inter-agent relations=no support, Handling conflicting signals=no support2026.05 | 94.1 | — | |
| DyLANMulti-agent collaboration=full support, Inter-agent relations=full support2026.05 | 94.04 | — | |
| VanillaMulti-agent collaboration=no support, Inter-agent relations=no support, Handling conflicting signals=no support2026.05 | 92.8 | — | |
| MANGO (Skip-2)Base LLM=GPT-4o-mini2026.05 | 92.7 | — | |
| MaASBase LLM=GPT-4o-mini2026.05 | 92.3 | — | |
| AFlowBase LLM=GPT-4o-mini2026.05 | 91.16 | — | |
| DyLANBase LLM=GPT-4o-mini2026.05 | 89.98 | — | |
| Self-RefineBase LLM=GPT-4o-mini2026.05 | 89.63 | — | |
| Plan-and-ExecuteBase LLM=GPT-4o-mini2026.05 | 89.52 | — | |
| LLM-DebateBase LLM=GPT-4o-mini2026.05 | 89.47 | — | |
| GPTSwarmBase LLM=GPT-4o-mini2026.05 | 89.14 | — | |
| AgentSquareBase LLM=GPT-4o-mini2026.05 | 87.62 | — | |
| SC (CoT×5)Base LLM=GPT-4o-mini2026.05 | 87.57 | — | |
| MultiPersonaBase LLM=GPT-4o-mini2026.05 | 87.5 | — | |
| IO (direct LLM invocation)Base LLM=GPT-4o-mini2026.05 | 87.45 | — | |
| CoTBase LLM=GPT-4o-mini2026.05 | 87.1 | — | |
| ADASBase LLM=GPT-4o-mini2026.05 | 86.12 | — | |
| LoRA-ProBase Model=Qwen3-8B, Evaluation Protocol=Non-Thinking 0-shot2023.08 | 84.5 | — | |
| LoRA-FABase Model=Qwen3-8B, Evaluation Protocol=Non-Thinking 0-shot2023.08 | 84.5 | — | |
| LoRABase Model=Qwen3-8B, Evaluation Protocol=Non-Thinking 0-shot2023.08 | 82.4 | — | |
| Uni-LoRAModel=Qwen-2.5-7B, # Params=524K2026.05 | 81.65 | — | |
| GPartModel=Qwen-2.5-7B, # Params=524K2026.05 | 81.43 | — | |
| GPartModel=Qwen-2.5-3B, # Params=524K2026.05 | 79.3 | — | |
| Uni-LoRAModel=Qwen-2.5-3B, # Params=524K2026.05 | 78.24 | — | |
| VanillaBase Model=Qwen3-8B, Evaluation Protocol=Reasoning mode2023.08 | 76.9 | — | |
| Uni-LoRAModel=Gemma-7B, # Params=524K2026.05 | 72.25 | — | |
| GPartModel=Gemma-7B, # Params=524K2026.05 | 71.42 | — | |
| LoRA-ProBase Model=DeepSeek-v2-lite-chat, Evaluation Protocol=Non-Thinking 0-shot2023.08 | 71.2 | — | |
| LoRA-FABase Model=DeepSeek-v2-lite-chat, Evaluation Protocol=Non-Thinking 0-shot2023.08 | 71.1 | — | |
| GPartModel=Average, # Params=//2026.05 | 69.66 | — | |
| GPartModel=Llama-3.1-8B, # Params=524K2026.05 | 69.45 | — | |
| Uni-LoRAModel=Average, # Params=//2026.05 | 69.26 | — | |
| Uni-LoRAModel=Llama-3.1-8B, # Params=524K2026.05 | 68.16 | — | |
| LoRABase Model=DeepSeek-v2-lite-chat, Evaluation Protocol=Non-Thinking 0-shot2023.08 | 66.5 | — | |
| VanillaBase Model=DeepSeek-v2-lite-chat, Evaluation Protocol=Reasoning mode2023.08 | 58.1 | — | |
| SEABase Model=LLaMA-3.2-1B-Instruct2025.05 | 58 | 7.28 | |
| BoN-64Base Model=LLaMA-3.2-1B-Instruct, N=642025.05 | 57 | 1.78 | |
| GPartModel=Qwen-2.5-0.5B, # Params=131K2026.05 | 46.7 | — | |
| Uni-LoRAModel=Qwen-2.5-0.5B, # Params=131K2026.05 | 46.02 | — | |
| BoN-32Base Model=LLaMA-3.2-1B-Instruct, N=322025.05 | 46 | 0.47 | |
| BoN-8Base Model=LLaMA-3.2-1B-Instruct, N=82025.05 | 42.5 | -1.22 | |
| CBSBase Model=LLaMA-3.2-1B-Instruct2025.05 | 37 | -0.53 | |
| SFTBase Model=LLaMA-3.2-1B-Instruct2025.05 | 32 | -1.44 | |
| RSBase Model=LLaMA-3.2-1B-Instruct2025.05 | 29.5 | -4.75 | |
| ARGSBase Model=LLaMA-3.2-1B-Instruct2025.05 | 20 | -4.28 |