Math Reasoning on GSM8K (test)
95.9AccuracyAgent-GWO
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Agent-GWOBackbone=GPT-4o-mini2026.04 | 95.9 | — | — | — | — | — | |
| AoTBackbone=GPT-4o-mini2026.04 | 95 | — | — | — | — | — | |
| ToTBackbone=GPT-4o-mini2026.04 | 94.9 | — | — | — | — | — | |
| GPT-4-Turbo-0409Few-shot CoT=4-shot2024.05 | 94.5 | — | — | — | — | — | |
| AFlowBackbone=GPT-4o-mini2026.04 | 93.5 | — | — | — | — | — | |
| MATH-SHEPHERDModel=DeepSeek-67B, Training Data=MetaMATH, Verification Samples=2562023.12 | 93.3 | — | — | — | — | — | |
| MATH-SHEPHERDModel=LLaMA2-70B, Training Data=MetaMATH, Verification Samples=2562023.12 | 93.2 | — | — | — | — | — | |
| GoTBackbone=GPT-4o-mini2026.04 | 93.2 | — | — | — | — | — | |
| Agent-GWOBackbone=Gemma-3-12b-it2026.04 | 92.8 | — | — | — | — | — | |
| ORMModel=DeepSeek-67B, Training Data=MetaMATH, Verification Samples=2562023.12 | 92.6 | — | — | — | — | — | |
| Self-Consistency + MATH-SHEPHERDModel=DeepSeek-67B, Training Data=MetaMATH, Verification Samples=2562023.12 | 92.5 | — | — | — | — | — | |
| Self-Consistency + MATH-SHEPHERDModel=LLaMA2-70B, Training Data=MetaMATH, Verification Samples=2562023.12 | 92.4 | — | — | — | — | — | |
| Self-Consistency + ORMModel=DeepSeek-67B, Training Data=MetaMATH, Verification Samples=2562023.12 | 92.4 | — | — | — | — | — | |
| Self-Consistency + ORMModel=LLaMA2-70B, Training Data=MetaMATH, Verification Samples=2562023.12 | 92 | — | — | — | — | — | |
| ORMModel=LLaMA2-70B, Training Data=MetaMATH, Verification Samples=2562023.12 | 91.8 | — | — | — | — | — | |
| AoTBackbone=Gemma-3-12b-it2026.04 | 91.4 | — | — | — | — | — | |
| MATH-SHEPHERDModel=LLemma-34B, Training Data=MetaMATH, Verification Samples=2562023.12 | 90.9 | — | — | — | — | — | |
| aPSFWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 90.87 | — | — | — | — | — | |
| GrIPSWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 90.8 | — | — | — | — | — | |
| CriSPOWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 90.13 | — | — | — | — | — | |
| CriSPOWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 90.11 | — | — | — | — | — | |
| aPSFWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 90.03 | — | — | — | — | — | |
| ORMModel=LLemma-34B, Training Data=MetaMATH, Verification Samples=2562023.12 | 90 | — | — | — | — | — | |
| ZERAWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 89.98 | — | — | — | — | — | |
| CoT-SC/n=5Backbone=GPT-4o-mini2026.04 | 89.9 | — | — | — | — | — | |
| Self-Consistency + MATH-SHEPHERDModel=LLemma-34B, Training Data=MetaMATH, Verification Samples=2562023.12 | 89.7 | — | — | — | — | — | |
| ProTeGiWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 89.7 | — | — | — | — | — | |
| ZERAWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 89.69 | — | — | — | — | — | |
| Self-Consistency + ORMModel=LLemma-34B, Training Data=MetaMATH, Verification Samples=2562023.12 | 89.6 | — | — | — | — | — | |
| ZERAWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 89.55 | — | — | — | — | — | |
| CriSPOWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 89.49 | — | — | — | — | — | |
| CriSPOWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 89.4 | — | — | — | — | — | |
| GoTBackbone=Gemma-3-12b-it2026.04 | 89.2 | — | — | — | — | — | |
| Agent-GWOBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 89.1 | — | — | — | — | — | |
| AFlowBackbone=Gemma-3-12b-it2026.04 | 89 | — | — | — | — | — | |
| Self-RefineBackbone=GPT-4o-mini2026.04 | 88.9 | — | — | — | — | — | |
| APEWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 88.83 | — | — | — | — | — | |
| OPROWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 88.7 | — | — | — | — | — | |
| AoTBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 88.5 | — | — | — | — | — | |
| ProTeGiWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 88.43 | — | — | — | — | — | |
| ToTBackbone=Gemma-3-12b-it2026.04 | 88.4 | — | — | — | — | — | |
| Self-ConsistencyModel=DeepSeek-67B, Training Data=MetaMATH, Verification Samples=2562023.12 | 88.2 | — | — | — | — | — | |
| GrIPSWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 88.16 | — | — | — | — | — | |
| Self-ConsistencyModel=LLaMA2-70B, Training Data=MetaMATH, Verification Samples=2562023.12 | 88 | — | — | — | — | — | |
| OPROWorker Model=Qwen2.5-7B-Instruct, Architect Model=Qwen3-8B2026.04 | 87.63 | — | — | — | — | — | |
| APEWorker Model=Qwen2.5-7B-Instruct, Architect Model=gpt-oss-120b2026.04 | 87.36 | — | — | — | — | — | |
| MAmmoTH2-8x7B-PlusParameter Size=8x7B, Training Stage=Continue trained with additional instruction datasets, Few-shot CoT=4-shot2024.05 | 86.4 | — | — | — | — | — | |
| Zero-shot CoTWorker Model=Qwen2.5-7B-Instruct, Optimization Protocol=No optimization (p_init)2026.04 | 86.22 | — | — | — | — | — | |
| aPSFWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 86.22 | — | — | — | — | — | |
| CoT-SC/n=5Backbone=Gemma-3-12b-it2026.04 | 85.9 | — | — | — | — | — | |
| aPSFWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 85.62 | — | — | — | — | — | |
| CoTBackbone=GPT-4o-mini2026.04 | 85.4 | — | — | — | — | — | |
| OPROWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 85.25 | — | — | — | — | — | |
| CNTP + SC (Ours + SC)Model=Llama-3.1-8B-Instruct, Reasoning Chain=Multiple Reasoning Chains, Decoding Strategy=Stochastic, paths=402025.07 | 85.2 | 18,400 | — | — | — | — | |
| OPROWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 84.95 | — | — | — | — | — | |
| SCModel=Llama-3.1-8B-Instruct, Reasoning Chain=Multiple Reasoning Chains, Decoding Strategy=Stochastic, paths=402025.07 | 84.8 | 6,030 | — | — | — | — | |
| GoTBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 84.8 | — | — | — | — | — | |
| MAmmoTH2-7B-PlusParameter Size=7B, Training Stage=Continue trained with additional instruction datasets, Few-shot CoT=4-shot2024.05 | 84.7 | — | — | — | — | — | |
| Self-RefineBackbone=Gemma-3-12b-it2026.04 | 84.6 | — | — | — | — | — | |
| ProTeGiWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 84.48 | — | — | — | — | — | |
| top-performing BoNRef. Model=Phi-3.5, n=20-200 (average)2026.05 | 84.47 | — | — | — | — | — | |
| GrIPSWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 84.28 | — | — | — | — | — | |
| SLOP Hard selectionRef. Model=Phi-3.5, n=20-200 (average)2026.05 | 84.27 | — | — | — | — | — | |
| Fixed weights (ω = 1)Ref. Model=Phi-3.5, n=20-200 (average)2026.05 | 84.27 | — | — | — | — | — | |
| ZERAWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 84.15 | — | — | — | — | — | |
| MAmmoTH2-8B-PlusParameter Size=8B, Training Stage=Continue trained with additional instruction datasets, Few-shot CoT=4-shot2024.05 | 84.1 | — | — | — | — | — | |
| AFlowBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 84 | — | — | — | — | — | |
| SLOP samplingRef. Model=Phi-3.5, n=20-200 (average)2026.05 | 83.91 | — | — | — | — | — | |
| Diagonal Covariance hard selectionRef. Model=Phi-3.5, n=20-200 (average)2026.05 | 83.77 | — | — | — | — | — | |
| Inverse Covariance hard selectionRef. Model=Phi-3.5, n=20-200 (average)2026.05 | 83.67 | — | — | — | — | — | |
| ToTBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 83.6 | — | — | — | — | — | |
| GrIPSWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 83.55 | — | — | — | — | — | |
| CoTBackbone=Gemma-3-12b-it2026.04 | 83.5 | — | — | — | — | — | |
| APEWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 83.48 | — | — | — | — | — | |
| Dense w/oPruning method=-, RM Blocks=0(0.0%), Backbone=Qwen 2.5-7B-Instruct2026.06 | 82.87 | — | — | — | — | — | |
| Zero-shot CoTWorker Model=Llama-3.1-8B-Instruct, Optimization Protocol=No optimization (p_init)2026.04 | 82.81 | — | — | — | — | — | |
| Self-ConsistencyModel=LLemma-34B, Training Data=MetaMATH, Verification Samples=2562023.12 | 82.6 | — | — | — | — | — | |
| Beam SearchModel=Llama-3.1-8B-Instruct, Reasoning Chain=Multiple Reasoning Chains, Decoding Strategy=Deterministic, beam=52025.07 | 82.3 | 740 | — | — | — | — | |
| CNTP (Ours*)Model=Llama-3.1-8B-Instruct, Reasoning Chain=Single Reasoning Chain, Decoding Strategy=Stochastic2025.07 | 81.6 | 460 | — | — | — | — | |
| Baseline reference samplingRef. Model=Phi-3.5, n=20-200 (average)2026.05 | 81.51 | — | — | — | — | — | |
| ProTeGiWorker Model=Llama-3.1-8B-Instruct, Architect Model=Qwen3-8B2026.04 | 81.07 | — | — | — | — | — | |
| APEWorker Model=Llama-3.1-8B-Instruct, Architect Model=gpt-oss-120b2026.04 | 80.2 | — | — | — | — | — | |
| CoT-SC/n=5Backbone=Qwen2.5-Coder-7B-Instruct2026.04 | 80 | — | — | — | — | — | |
| Greedy DecodingModel=Llama-3.1-8B-Instruct, Reasoning Chain=Single Reasoning Chain, Decoding Strategy=Deterministic2025.07 | 79.8 | 150 | — | — | — | — | |
| Stochastic Decoding*Model=Llama-3.1-8B-Instruct, Reasoning Chain=Single Reasoning Chain, Decoding Strategy=Stochastic2025.07 | 79.4 | 150 | — | — | — | — | |
| Self-RefineBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 79 | — | — | — | — | — | |
| MinervaModel Parameters=540B, Majority Voting (maj1@k)=true, k samples=402022.06 | 78.5 | — | — | — | — | — | |
| CoTBackbone=Qwen2.5-Coder-7B-Instruct2026.04 | 77.5 | — | — | — | — | — | |
| InherNetParameters=4.2B, Base Model=LLaMA-2-7B, Number of experts (H)=4, Rank (r)=2562026.02 | 77.19 | — | — | — | — | — | |
| SLOP Hard selectionRef. Model=Qwen3, n=20-200 (average)2026.05 | 76.93 | — | — | — | — | — | |
| Student (Self-Distillation)Parameters=7B, Base Model=LLaMA-2-7B2026.02 | 76.88 | — | — | — | — | — | |
| Diagonal Covariance hard selectionRef. Model=Qwen3, n=20-200 (average)2026.05 | 76.49 | — | — | — | — | — | |
| TeacherParameters=7B, Base Model=LLaMA-2-7B2026.02 | 76.34 | — | — | — | — | — | |
| Inverse Covariance hard selectionRef. Model=Qwen3, n=20-200 (average)2026.05 | 76.16 | — | — | — | — | — | |
| SLOP samplingRef. Model=Qwen3, n=20-200 (average)2026.05 | 76.05 | — | — | — | — | — | |
| MAmmoTH2-34BParameter Size=34B, Training Stage=Trained only with WEBINSTRUCT, Few-shot CoT=4-shot2024.05 | 75.6 | — | — | — | — | — | |
| top-performing BoNRef. Model=Qwen3, n=20-200 (average)2026.05 | 75.51 | — | — | — | — | — | |
| MAmmoTH2-8x7BParameter Size=8x7B, Training Stage=Trained only with WEBINSTRUCT, Few-shot CoT=4-shot2024.05 | 75.4 | — | — | — | — | — | |
| PaLM 540B maj1@40 (Published SOTA)Majority Voting (maj1@k)=true, k samples=402022.06 | 74.4 | — | — | — | — | — | |
| CoT + Self-ConsistencyBackbone=PaLM 540B, Protocol=few-shot2022.10 | 74.4 | — | — | — | — | — |