Mathematical Reasoning on MATH500 1.0 (test)
96.9AccuracyA3PO
Evaluation Results
| Method | Links | |
|---|---|---|
| A3POModel=Deepseek-R1-Distill-Qwen-7B2025.12 | 96.9 | |
| Lp-RegModel=Deepseek-R1-Distill-Qwen-7B2025.12 | 96.2 | |
| DAPO w/ Fork TokensModel=Deepseek-R1-Distill-Qwen-7B2025.12 | 95.8 | |
| W-REINFORCEModel=Deepseek-R1-Distill-Qwen-7B2025.12 | 95.7 | |
| DAPOModel=Deepseek-R1-Distill-Qwen-7B2025.12 | 95.5 | |
| GRPOModel=Deepseek-R1-Distill-Qwen-7B2025.12 | 95.2 | |
| A3POModel=Qwen3-8B-Base2025.12 | 91.3 | |
| A3POModel=Qwen2.5-7B-Math2025.12 | 90.4 | |
| Lp-RegModel=Qwen3-8B-Base2025.12 | 87.4 | |
| W-REINFORCEModel=Qwen2.5-7B-Math2025.12 | 87.3 | |
| Lp-RegModel=Qwen2.5-7B-Math2025.12 | 87.1 | |
| NEMOTRON-NANO-12B-V2 (RLP + Post)Pre-training objective=RLP, Post-training pipeline=SFT + RLVR, Evaluation protocol=Pass@1 (average of 8 runs)2025.09 | 87.05 | |
| W-REINFORCEModel=Qwen3-8B-Base2025.12 | 86.9 | |
| DAPO w/ Fork TokensModel=Qwen2.5-7B-Math2025.12 | 86.8 | |
| DAPO w/ Fork TokensModel=Qwen3-8B-Base2025.12 | 86.2 | |
| DAPOModel=Qwen2.5-7B-Math2025.12 | 85.4 | |
| DAPOModel=Qwen3-8B-Base2025.12 | 84.5 | |
| GRPOModel=Qwen2.5-7B-Math2025.12 | 83.8 | |
| NEMOTRON-NANO-12B-V2 (Base + Post)Pre-training objective=Next-token prediction, Post-training pipeline=SFT + RLVR, Evaluation protocol=Pass@1 (average of 8 runs)2025.09 | 83.47 | |
| GRPOModel=Qwen3-8B-Base2025.12 | 82.3 | |
| NEMOTRON-NANO-12B-V2 (Base)Pre-training objective=Next-token prediction, Post-training pipeline=None, Evaluation protocol=Pass@1 (average of 8 runs)2025.09 | 79.95 | |
| NEMOTRON-NANO-12B-V2 (RLP)Pre-training objective=RLP, Post-training pipeline=None, Evaluation protocol=Pass@1 (average of 8 runs)2025.09 | 78.68 | |
| polyGRPOBackbone=Qwen2.5-7B-Instruct2026.04 | 64.94 | |
| GRPOBackbone=Qwen2.5-7B-Instruct2026.04 | 63.2 | |
| LIDRBackbone=Qwen2.5-7B-Instruct2026.04 | 62.46 | |
| MAPOBackbone=Qwen2.5-7B-Instruct2026.04 | 61.2 | |
| BaseBackbone=Qwen2.5-7B-Instruct2026.04 | 57.89 | |
| xRFTBackbone=Qwen2.5-7B-Instruct2026.04 | 53.12 | |
| Llama3.1-8B-Instruct# data=N.A.2026.02 | 51.9 | |
| SFT-Decomp + Curriculum# data=4500, backbone=Qwen2.5-1.5B2026.02 | 51.6 | |
| SFT-Decomp# data=4500, backbone=Qwen2.5-1.5B2026.02 | 50.8 | |
| SFT-MuggleMath# data=147787, backbone=Qwen2.5-1.5B2026.02 | 50.4 | |
| SFT-Direct Distillation# data=360, backbone=Qwen2.5-1.5B2026.02 | 49.6 | |
| SFT# data=360, backbone=Qwen2.5-1.5B2026.02 | 48.4 | |
| SFT (full dataset)# data=7500, backbone=Qwen2.5-1.5B2026.02 | 47.6 | |
| Base# data=N.A., backbone=Qwen2.5-1.5B2026.02 | 47.2 | |
| MiniCPM3-4B# data=N.A.2026.02 | 46.6 | |
| Gemma2-9B-Instruct# data=N.A.2026.02 | 44.3 | |
| Gemma2-27B# data=N.A.2026.02 | 42.7 | |
| Qwen2.5-3B# data=N.A.2026.02 | 42.6 | |
| Llama-3-70B# data=N.A.2026.02 | 42.5 | |
| TABOMSFT Training Domain=Code SFT (Ling-Coder-SFT)2026.05 | 42.4 | |
| Mixtral-8x22B# data=N.A.2026.02 | 41.7 | |
| SFT-SDSFT Training Domain=Code SFT (Ling-Coder-SFT)2026.05 | 41.6 | |
| TABOMSFT Training Domain=Math SFT (MixChain-Z-PRM12K)2026.05 | 41.1 | |
| No-SFTSFT Training Domain=None (Base Model)2026.05 | 39.8 | |
| SFT-SDSFT Training Domain=Math SFT (MixChain-Z-PRM12K)2026.05 | 39.8 | |
| SFT-GTSFT Training Domain=Math SFT (MixChain-Z-PRM12K)2026.05 | 37.4 | |
| SFT-MetaMATH-Aug# data=2638, backbone=Qwen2.5-1.5B2026.02 | 37.2 | |
| Qwen2.5-1.5B# data=N.A.2026.02 | 35 | |
| SFT-GTSFT Training Domain=Code SFT (Ling-Coder-SFT)2026.05 | 32.4 | |
| polyGRPOBackbone=Llama3-8B-Instruct2026.04 | 26.71 | |
| BaseBackbone=Llama3-8B-Instruct2026.04 | 26 | |
| MAPOBackbone=Llama3-8B-Instruct2026.04 | 25.26 | |
| GRPOBackbone=Llama3-8B-Instruct2026.04 | 24.43 | |
| LIDRBackbone=Llama3-8B-Instruct2026.04 | 24.31 | |
| xRFTBackbone=Llama3-8B-Instruct2026.04 | 23.37 |