Math Word Problem Solving on GSM8K
96.8AccuracyLlama-3.1-405B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Llama-3.1-405BModel Type=Instruct, Parameters=405B2025.02 | 96.8 | — | |
| Qwen2.5-72BModel Type=Instruct, Parameters=72B2025.02 | 95.8 | — | |
| Primitives-based MASBackbone=Qwen3-14B2026.02 | 95.6 | 11.8 | |
| RAPSNumber of agents=52026.02 | 95.4 | — | |
| Qwen2.5-VL-72BModel Type=Instruct, Parameters=72B2025.02 | 95.3 | — | |
| LatentMASBackbone=Qwen3-14B2026.02 | 95.2 | 11.4 | |
| VotingBackbone=Qwen3-14B2026.02 | 95.2 | 11.4 | |
| LatentMASModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 95.2 | — | |
| Llama-3.1-70BModel Type=Instruct, Parameters=70B2025.02 | 95.1 | — | |
| VotingModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 95 | — | |
| Primitives-based MASModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 95 | — | |
| PlanningBackbone=Qwen3-14B2026.02 | 94.6 | 10.8 | |
| ReviewModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 94.6 | — | |
| ReviewBackbone=Qwen3-14B2026.02 | 94.4 | 10.6 | |
| Primitives-based MASModel=Qwen3-8B2026.02 | 94.2 | — | |
| AFlowNumber of agents=52026.02 | 94.1 | — | |
| TextMASBackbone=Qwen3-14B2026.02 | 93.8 | 10 | |
| LatentMASModel=Qwen3-8B2026.02 | 93.8 | — | |
| SingleModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 93.8 | — | |
| TextMASModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 93.8 | — | |
| PlanningModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 93.8 | — | |
| VotingModel=DeepSeek-R1-Distill Llama-70B2026.02 | 93.8 | — | |
| Primitives-based MASModel=DeepSeek-R1-Distill Llama-70B2026.02 | 93.8 | — | |
| PuppeteerNumber of agents=52026.02 | 93.3 | — | |
| Qwen2-72BModel Type=Instruct, Parameters=72B2025.02 | 93.2 | — | |
| G-DesignerNumber of agents=52026.02 | 93.2 | — | |
| ReviewModel=Qwen3-8B2026.02 | 93.2 | — | |
| PlanningModel=Qwen3-8B2026.02 | 93.2 | — | |
| TextMASModel=DeepSeek-R1-Distill Llama-70B2026.02 | 93.2 | — | |
| GPTSwarmNumber of agents=52026.02 | 92.7 | — | |
| MAS-ZeroNumber of agents=52026.02 | 92.6 | — | |
| ComplexCoTNumber of agents=12026.02 | 92.5 | — | |
| AutoAgentsNumber of agents=52026.02 | 92.5 | — | |
| SCNumber of agents=12026.02 | 92.4 | — | |
| LLM-DebateNumber of agents=52026.02 | 92.4 | — | |
| SingleModel=DeepSeek-R1-Distill Llama-70B2026.02 | 92.4 | — | |
| PlanningModel=DeepSeek-R1-Distill Llama-70B2026.02 | 92.4 | — | |
| AgentPruneNumber of agents=52026.02 | 92.3 | — | |
| TextMASModel=Qwen3-8B2026.02 | 92.3 | — | |
| CoTNumber of agents=12026.02 | 92.1 | — | |
| RandomNumber of agents=52026.02 | 92 | — | |
| StarNumber of agents=52026.02 | 91.9 | — | |
| VotingModel=Qwen3-8B2026.02 | 91.8 | — | |
| ReviewModel=DeepSeek-R1-Distill Llama-70B2026.02 | 91.8 | — | |
| ChainNumber of agents=52026.02 | 91.7 | — | |
| Vanilla IONumber of agents=1, Backbone=GPT-4o-mini2026.02 | 91.6 | — | |
| Primitives-based MASBackbone=Qwen3-4B2026.02 | 91.6 | 9.2 | |
| MaASNumber of agents=52026.02 | 91.4 | — | |
| LLM-BlenderNumber of agents=52026.02 | 91.3 | — | |
| TreeNumber of agents=52026.02 | 90.7 | — | |
| VotingBackbone=Qwen3-4B2026.02 | 90.4 | 8 | |
| PlanningBackbone=Qwen3-4B2026.02 | 90 | 7.6 | |
| TextMASBackbone=Qwen3-4B2026.02 | 89.8 | 7.4 | |
| ReviewBackbone=Qwen3-4B2026.02 | 88.6 | 6.2 | |
| DeepSeekMath-7B-RLTraining Method=RL, Backbone=DeepSeekMath-7B2024.06 | 88.4 | — | |
| DART-Math-DSMath-7BTraining Method=SFT, Data Selection Strategy=Uniform, Backbone=DeepSeekMath-7B2024.06 | 88.2 | — | |
| LatentMASBackbone=Qwen3-4B2026.02 | 88.2 | 5.8 | |
| DART-Math-DSMath-7BTraining Method=SFT, Data Selection Strategy=Prop2Diff, Backbone=DeepSeekMath-7B2024.06 | 86.8 | — | |
| Hard-Routed MoR-LoRABackbone=Llama-8B, # Trainable Parameters=≈ 109M2026.06 | 84.69 | — | |
| SingleBackbone=Qwen3-14B2026.02 | 83.8 | — | |
| SingleBackbone=Qwen3-4B2026.02 | 82.4 | — | |
| LoRAMixer TopK=2 NormalizedBackbone=Llama-8B, # Trainable Parameters=≈ 1.133B2026.06 | 82.15 | — | |
| SingleModel=Qwen3-8B2026.02 | 81.1 | — | |
| LoRAMixer TopK=1Backbone=Llama-8B, # Trainable Parameters=≈ 1.133B2026.06 | 79.51 | — | |
| LatentMASModel=DeepSeek-R1-Distill Llama-70B2026.02 | 78.6 | — | |
| Hard-Routed MoR-LoRABackbone=Llama-3B, # Trainable Parameters=≈ 73M2026.06 | 73.62 | — | |
| LoRAMixer TopK=2 NormalizedBackbone=Llama-3B, # Trainable Parameters=≈ 606M2026.06 | 69.67 | — | |
| LoRAMixer TopK=1Backbone=Llama-3B, # Trainable Parameters=≈ 606M2026.06 | 63.53 | — | |
| GRADIENTSPACEBackbone Model=LLaMA-2-7B, initialization=SVD init2025.12 | 54.7 | — | |
| COLLATEBackbone=open-llama-v2-7b2025.06 | 53.48 | — | |
| K-means + LoRA (Mixture of LoRA)Backbone Model=LLaMA-2-7B2025.12 | 53.1 | — | |
| ELREABackbone Model=LLaMA-2-7B2025.12 | 52.4 | — | |
| FinetuningBackbone Model=LLaMA-2-7B2025.12 | 50.8 | — | |
| RPOChain-of-thought=false, Few-shots=false, Backbone=Gemma2024.05 | 49.9 | — | |
| SPINCategory=Training-driven Rationale Enhancement, Backbone=open-llama-v2-7b2025.06 | 48.41 | — | |
| LoRA FinetuningBackbone Model=LLaMA-2-7B2025.12 | 47.6 | — | |
| zephyr-gemma-7bChain-of-thought=false, Few-shots=false, Note=Officially released model trained by DPO2024.05 | 47.3 | — | |
| GRADIENTSPACEBackbone Model=LLaMA-3.2-1B, initialization=SVD init2025.12 | 46.7 | — | |
| zephyr-7b-gemma-sftChain-of-thought=false, Few-shots=false, Backbone=Gemma2024.05 | 45.4 | — | |
| DPOChain-of-thought=false, Few-shots=false, Backbone=Gemma2024.05 | 45.3 | — | |
| K-means + LoRA (Mixture of LoRA)Backbone Model=LLaMA-3.2-1B2025.12 | 44.8 | — | |
| ELREABackbone Model=LLaMA-3.2-1B2025.12 | 43.2 | — | |
| FinetuningBackbone Model=LLaMA-3.2-1B2025.12 | 41.5 | — | |
| LoRA FinetuningBackbone Model=LLaMA-3.2-1B2025.12 | 40.7 | — | |
| Model MergingBackbone Model=LLaMA-2-7B2025.12 | 40.7 | — | |
| Zero-shot prompting (Pretrained)Backbone Model=LLaMA-2-7B2025.12 | 40.6 | — | |
| Self-Rewarding LMsCategory=Training-driven Rationale Enhancement, Backbone=open-llama-v2-7b2025.06 | 39.13 | — | |
| Distilling Step-by-StepCategory=Task-specific Supervised Fine-Tuning, Backbone=open-llama-v2-7b2025.06 | 38.41 | — | |
| Random clustering + LoRABackbone Model=LLaMA-2-7B2025.12 | 35.4 | — | |
| Zero-shot prompting (Pretrained)Backbone Model=LLaMA-3.2-1B2025.12 | 31.3 | — | |
| Random clustering + LoRABackbone Model=LLaMA-3.2-1B2025.12 | 31.2 | — | |
| open-llama-7b-v2 SFTCategory=Task-specific Supervised Fine-Tuning, Backbone=open-llama-v2-7b2025.06 | 30.76 | — | |
| Model MergingBackbone Model=LLaMA-3.2-1B2025.12 | 30.6 | — | |
| Exchange-of-ThoughtCategory=Prompt-based Rationale Refinement, Backbone=open-llama-v2-7b2025.06 | 26.8 | — | |
| Tree-of-ThoughtCategory=Prompt-based Rationale Refinement, Backbone=open-llama-v2-7b2025.06 | 17.31 | — | |
| SFed-LoRARank=512, Backbone=LLaMA2-7B2026.03 | 17.22 | — | |
| SFed-LoRARank=8, Backbone=LLaMA2-7B2026.03 | 17.08 | — | |
| FedSA-rsLoRARank=8, Backbone=LLaMA2-7B2026.03 | 16.67 | — | |
| RoLoRARank=512, Backbone=LLaMA2-7B2026.03 | 16.66 | — | |
| SFed-LoRARank=32, Backbone=LLaMA2-7B2026.03 | 16.66 | — |