Math Word Problem Solving on GSM8K (Accuracy)
95.3AccuracyOracle
Evaluation results
| Method | Links | |
|---|---|---|
| OracleModel=Gemma2-9B, Black-Box Friendly=true2026.04 | 95.3 | |
| AgentReviveVar. NS=True, Flex. State=True, Base model=Qwen2.5-72B-Instruct2026.05 | 94.87 | |
| OracleModel=Qwen2.5-7B, Black-Box Friendly=true2026.04 | 94.6 | |
| OracleModel=Llama3.1-8B, Black-Box Friendly=true2026.04 | 94.6 | |
| GPT-5 nanoScale=nano2026.05 | 94.6 | |
| Qwen3-8BParameters=8B2026.05 | 94.3 | |
| AgentDropoutVar. NS=True, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.05 | 93.75 | |
| SC (CoT)Var. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.05 | 93.67 | |
| AgentPruneVar. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.05 | 93.67 | |
| MASround=1Var. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.05 | 93.52 | |
| MASround=TVar. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.05 | 93.28 | |
| OracleModel=Qwen2.5-3B, Black-Box Friendly=true2026.04 | 93.1 | |
| ARG-DesignerVar. NS=True, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.05 | 92.98 | |
| phi-balancingModel=Moonlight-16B-A3B-Instruct2026.05 | 92.92 | |
| gpt-oss-20bParameters=20B2026.05 | 92.9 | |
| Frozen checkpointModel=Moonlight-16B-A3B-Instruct2026.05 | 92.84 | |
| OracleModel=Llama3.2-3B, Black-Box Friendly=true2026.04 | 92.6 | |
| G-DesignerVar. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.05 | 92.5 | |
| CoTVar. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.05 | 92.19 | |
| AutogenVar. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.05 | 92.17 | |
| AgentVerseVar. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.05 | 91.59 | |
| ST-MoEModel=Moonlight-16B-A3B-Instruct2026.05 | 91.37 | |
| VanillaVar. NS=False, Flex. State=False, Base model=Qwen2.5-72B-Instruct2026.05 | 91.02 | |
| gemma-3-12b-itParameters=12B2026.05 | 90.7 | |
| GreedyModel=Qwen2.5-7B, Black-Box Friendly=true2026.04 | 88.3 | |
| GreedyModel=Llama3.1-8B, Black-Box Friendly=true2026.04 | 88.3 | |
| Qwen3-4BParameters=4B2026.05 | 87.5 | |
| EngGPT2-16B-A3BParameters=16B2026.05 | 86.4 | |
| Llama-3.1-8BParameters=8B2026.05 | 86.1 | |
| OVMBase Model=Mistral, Size=7B+7B2026.04 | 84.7 | |
| ToRA-CodeBase Model=LLaMA-2, Size=70B, Synthetic Data=TORA-CORPUS (16K)2026.04 | 84.3 | |
| RCSprobModel=Llama3.2-3B, Black-Box Friendly=false2026.04 | 84.1 | |
| AbelBase Model=LLaMA-2, Size=70B, Synthetic Data=Unreleased2026.04 | 83.9 | |
| Ministral-3-8BParameters=8B2026.05 | 83.7 | |
| I-DPO + MaPPOModel=Llama-3-8B-Instruct2025.07 | 82.4 | |
| I-DPO + MaPPOBase Model=Llama-3-8B-Instruct, DPO Mode=iterative2026.05 | 82.4 | |
| MetaMathBase Model=LLaMA-2, Size=70B, Synthetic Data=MetaMathQA (395K)2026.04 | 82.1 | |
| I-DPO + MaPPOModel=Qwen2.5-7B-Instruct2025.07 | 82 | |
| ITModel=Qwen2.5-7B-Instruct2025.07 | 81.7 | |
| WizardMathBase Model=LLaMA-2, Size=70B, Synthetic Data=Evol-Instruct2026.04 | 81.6 | |
| BaselineFine-tuning=Prior to fine-tuning2026.04 | 81.52 | |
| Phi-GSM+VBase Model=Phi-1.5, Size=1.3B+1.3B, Synthetic Data=TinyGSM (1.3M)2026.04 | 81.5 | |
| gemma-3-4b-itParameters=4B2026.05 | 81.5 | |
| I-DPOModel=Llama-3-8B-Instruct2025.07 | 81.3 | |
| I-DPOBase Model=Llama-3-8B-Instruct, DPO Mode=iterative2026.05 | 81.3 | |
| TLPOFine-tuning=After fine-tuning2026.04 | 80.99 | |
| ToRA-CodeBase Model=LLaMA-2, Size=34B, Synthetic Data=TORA-CORPUS (16K)2026.04 | 80.7 | |
| RCSmedoidModel=Llama3.2-3B, Black-Box Friendly=true2026.04 | 80.5 | |
| RCSuniModel=Llama3.2-3B, Black-Box Friendly=true2026.04 | 80.5 | |
| DPO + MaPPOModel=Qwen2.5-7B-Instruct2025.07 | 80.1 | |
| GreedyModel=Llama3.2-3B, Black-Box Friendly=true2026.04 | 79.7 | |
| TLPOFine-tuning=After fine-tuning2026.04 | 79.55 | |
| DPO + MaPPOModel=Llama-3-8B-Instruct2025.07 | 79.5 | |
| DPO + MaPPOBase Model=Llama-3-8B-Instruct, DPO Mode=offline2026.05 | 79.5 | |
| RCSfreqModel=Llama3.2-3B, Black-Box Friendly=true2026.04 | 79.4 | |
| DPOFine-tuning=After fine-tuning2026.04 | 78.57 | |
| CEModel=Llama3.2-3B, Black-Box Friendly=false2026.04 | 78.5 | |
| SFTTraining Dataset=s1K, Tuning Method=LoRA Tuning, Base Model=LLaDA-8B-Instruct, Evaluation Protocol=0-shot2025.09 | 78.5 | |
| BaselineFine-tuning=Prior to fine-tuning2026.04 | 78.48 | |
| Llama-3.2-3BParameters=3B2026.05 | 78.1 | |
| ORPOFine-tuning=After fine-tuning2026.04 | 78.09 | |
| SCModel=Llama3.2-3B, Black-Box Friendly=true2026.04 | 78 | |
| MetaMath-MistralBase Model=Mistral, Size=7B, Synthetic Data=MetaMathQA (395K)2026.04 | 77.8 | |
| SFTTraining Dataset=s1K-1.1, Tuning Method=LoRA Tuning, Base Model=LLaDA-8B-Instruct, Evaluation Protocol=0-shot2025.09 | 77.7 | |
| GIFTTraining Dataset=openr1-3k, Tuning Method=LoRA Tuning, Base Model=LLaDA-8B-Instruct, Evaluation Protocol=0-shot2025.09 | 77.7 | |
| GIFTTraining Dataset=s1K-1.1, Tuning Method=LoRA Tuning, Base Model=LLaDA-8B-Instruct, Evaluation Protocol=0-shot2025.09 | 77.6 | |
| Moonlight-16B-A3BParameters=16B2026.05 | 77.4 | |
| GIFTTraining Dataset=s1K, Tuning Method=LoRA Tuning, Base Model=LLaDA-8B-Instruct, Evaluation Protocol=0-shot2025.09 | 77.3 | |
| SFTTraining Dataset=openr1-3k, Tuning Method=LoRA Tuning, Base Model=LLaDA-8B-Instruct, Evaluation Protocol=0-shot2025.09 | 77.1 | |
| LLaDA-8B-InstructEvaluation Protocol=0-shot2025.09 | 76.8 | |
| MAMmoTHBase Model=LLaMA-2, Size=70B, Synthetic Data=MathInstruct (260K)2026.04 | 75.9 | |
| ToRA-CodeBase Model=LLaMA-2, Size=13B, Synthetic Data=TORA-CORPUS (16K)2026.04 | 75.8 | |
| RCSprobModel=Llama3.1-8B, Black-Box Friendly=false2026.04 | 75.6 | |
| DPOModel=Llama-3-8B-Instruct2025.07 | 75.5 | |
| DPOBase Model=Llama-3-8B-Instruct, DPO Mode=offline2026.05 | 75.5 | |
| AMRBase Model=Qwen2.5-Math, Size=7B, Synthetic Data=No2026.04 | 75.28 | |
| RCSprobModel=Gemma2-9B, Black-Box Friendly=false2026.04 | 75.1 | |
| DPOFine-tuning=After fine-tuning2026.04 | 75.06 | |
| Phi-GSMBase Model=Phi-2, Size=2.7B, Synthetic Data=TinyGSM (1.3M)2026.04 | 74.3 | |
| NLLModel=Llama3.2-3B, Black-Box Friendly=false2026.04 | 74.3 | |
| ANLLModel=Llama3.2-3B, Black-Box Friendly=false2026.04 | 73.9 | |
| RCSuniModel=Gemma2-9B, Black-Box Friendly=true2026.04 | 73.8 | |
| OVMBase Model=LLaMA-2, Size=7B+7B2026.04 | 73.7 | |
| ITModel=Llama-3-8B-Instruct2025.07 | 73.4 | |
| ITBase Model=Llama-3-8B-Instruct2026.05 | 73.4 | |
| I-DPOModel=Qwen2.5-7B-Instruct2025.07 | 73.2 | |
| ORPOFine-tuning=After fine-tuning2026.04 | 73.14 | |
| CEModel=Gemma2-9B, Black-Box Friendly=false2026.04 | 73.1 | |
| ANLLModel=Gemma2-9B, Black-Box Friendly=false2026.04 | 73.1 | |
| SCModel=Gemma2-9B, Black-Box Friendly=true2026.04 | 72.9 | |
| RCSmedoidModel=Gemma2-9B, Black-Box Friendly=true2026.04 | 72.9 | |
| RCSuniModel=Llama3.1-8B, Black-Box Friendly=true2026.04 | 72.8 | |
| ToRA-CodeBase Model=LLaMA-2, Size=7B, Synthetic Data=TORA-CORPUS (16K)2026.04 | 72.6 | |
| RCSfreqModel=Gemma2-9B, Black-Box Friendly=true2026.04 | 72.6 | |
| SEGOBase Model=CodeLLaMA, Size=13B, Synthetic Data=GSM8K+MATH+AQuA2026.04 | 72.5 | |
| RCSmedoidModel=Llama3.1-8B, Black-Box Friendly=true2026.04 | 71.4 | |
| Phi-GSM+V (350M)Base Model=Phi-1.5-small, Size=350M+350M, Synthetic Data=TinyGSM (1.3M)2026.04 | 71.3 | |
| DPOModel=Qwen2.5-7B-Instruct2025.07 | 71.3 | |
| MetaMathBase Model=LLaMA-2, Size=13B, Synthetic Data=MetaMathQA (395K)2026.04 | 70.8 | |
| Frozen checkpointModel=DeepSeek-V2-Lite2026.05 | 69.2 |