Mathematical Reasoning on ASDiv (Pass@1)
95.3Pass@1Baseline
Evaluation Results
| Method | Links | |
|---|---|---|
| BaselineBase Model=Qwen2.5-Math-7B-Instruct, Evaluation Protocol=Zero-shot2025.10 | 95.3 | |
| SFTBase Model=Qwen2.5-Math-7B-Instruct, Evaluation Protocol=Zero-shot2025.10 | 95.2 | |
| LightRBase Model=Qwen2.5-Math-7B-Instruct, Evaluation Protocol=Zero-shot2025.10 | 95.2 | |
| Qwen2.5-7B + GRPO w/ VERL.Base Model=Qwen2.5-7B, RL Framework=GRPO, VERL Integration=true2025.09 | 95 | |
| Qwen2.5-7B + PPOBase Model=Qwen2.5-7B, RL Framework=PPO, VERL Integration=false2025.09 | 94.9 | |
| Qwen2.5-7B + GRPOBase Model=Qwen2.5-7B, RL Framework=GRPO, VERL Integration=false2025.09 | 94.8 | |
| Qwen2.5-7B + PPO w/ VERL.Base Model=Qwen2.5-7B, RL Framework=PPO, VERL Integration=true2025.09 | 94.8 | |
| BaselineBase Model=Qwen2.5-Math-1.5B-Instruct, Evaluation Protocol=Zero-shot2025.10 | 94.7 | |
| SFTBase Model=Qwen2.5-Math-1.5B-Instruct, Evaluation Protocol=Zero-shot2025.10 | 94.7 | |
| LightRBase Model=Qwen2.5-Math-1.5B-Instruct, Evaluation Protocol=Zero-shot2025.10 | 94.1 | |
| Qwen2.5-7BBase Model=Qwen2.5-7B, RL Framework=None, VERL Integration=false2025.09 | 91.4 | |
| GPT-4 (0613)Zero-shot without demonstrations=false2024.03 | 91.3 | |
| LEMMA (w/ MetaMath)Backbone=DeepSeekMath-7B, # Samples=403.59k2025.03 | 90.4 | |
| LEMMABackbone=DeepSeekMath-7B, # Samples=88.90k2025.03 | 89.8 | |
| Llama-3.2-3B-Instruct + GRPO w/ VERL.Base Model=Llama-3.2-3B-Instruct, RL Framework=GRPO, VERL Integration=true2025.09 | 89.3 | |
| KPMath-PlusBase=Qwen1.5, Size=72B, Zero-shot without demonstrations=true2024.03 | 89.2 | |
| KPMath-PlusBase=Llama-2, Size=70B, Zero-shot without demonstrations=true2024.03 | 89 | |
| KPMath-PlusBase=DSMath, Size=7B, Zero-shot without demonstrations=true2024.03 | 88.9 | |
| Llama-3.2-3B-Instruct + GRPOBase Model=Llama-3.2-3B-Instruct, RL Framework=GRPO, VERL Integration=false2025.09 | 88.8 | |
| GPTAugBackbone=DeepSeekMath-7B, # Samples=88.62k2025.03 | 88.7 | |
| Llama-3.2-3B-Instruct + PPO w/ VERL.Base Model=Llama-3.2-3B-Instruct, RL Framework=PPO, VERL Integration=true2025.09 | 88.7 | |
| Llama-3.2-3B-Instruct + PPOBase Model=Llama-3.2-3B-Instruct, RL Framework=PPO, VERL Integration=false2025.09 | 87.9 | |
| MetaMathBackbone=DeepSeekMath-7B, # Samples=394.99k2025.03 | 87.8 | |
| KPMath-PlusBase=Llemma, Size=34B, Zero-shot without demonstrations=true2024.03 | 87.5 | |
| LightRBase Model=DeepSeek-R1-Distill-Qwen-1.5B, Evaluation Protocol=Zero-shot2025.10 | 87.5 | |
| SFTBase Model=DeepSeek-R1-Distill-Qwen-1.5B, Evaluation Protocol=Zero-shot2025.10 | 87.4 | |
| ChatGPTZero-shot without demonstrations=false2024.03 | 87.3 | |
| LEMMA (w/ MetaMath)Backbone=LLaMA3-8B, # Samples=403.59k2025.03 | 87.1 | |
| KPMath-PlusBase=Mistral, Size=7B, Zero-shot without demonstrations=true2024.03 | 86.7 | |
| RFTBackbone=DeepSeekMath-7B, # Samples=86.52k2025.03 | 85.2 | |
| BaselineBase Model=DeepSeek-R1-Distill-Qwen-1.5B, Evaluation Protocol=Zero-shot2025.10 | 84.9 | |
| LEMMABackbone=LLaMA3-8B, # Samples=88.90k2025.03 | 84.2 | |
| KPMath-PlusBase=Llama-2, Size=13B, Zero-shot without demonstrations=true2024.03 | 83.2 | |
| RefAug-90kBackbone=DeepSeekMath-7B, # Samples=89.92k2025.03 | 82.4 | |
| RefAugBackbone=LLaMA3-8B, # Samples=29.94k2025.03 | 82.3 | |
| ISCBackbone=DeepSeekMath-7B, # Samples=86.78k2025.03 | 82.2 | |
| RefAug-90kBackbone=LLaMA3-8B, # Samples=89.92k2025.03 | 82.1 | |
| MetaMathBackbone=LLaMA3-8B, # Samples=394.99k2025.03 | 81.9 | |
| GPTAugBackbone=LLaMA3-8B, # Samples=88.62k2025.03 | 81.2 | |
| RefAugBackbone=DeepSeekMath-7B, # Samples=29.94k2025.03 | 81.2 | |
| ISCBackbone=LLaMA3-8B, # Samples=86.78k2025.03 | 81.1 | |
| SFTBackbone=DeepSeekMath-7B, # Samples=14.97k2025.03 | 80.9 | |
| LightRBase Model=Qwen2.5-Math-7B, Evaluation Protocol=Zero-shot2025.10 | 80.6 | |
| LightRBase Model=Qwen2.5-Math-1.5B, Evaluation Protocol=Zero-shot2025.10 | 79.8 | |
| SFTBase Model=Qwen2.5-Math-7B, Evaluation Protocol=Zero-shot2025.10 | 76.6 | |
| RFTBackbone=LLaMA3-8B, # Samples=86.52k2025.03 | 74.8 | |
| Llama-3.2-3B-InstructBase Model=Llama-3.2-3B-Instruct, RL Framework=None, VERL Integration=false2025.09 | 74.6 | |
| BaselineBase Model=Qwen2.5-Math-7B, Evaluation Protocol=Zero-shot2025.10 | 72.7 | |
| SFTBackbone=LLaMA3-8B, # Samples=14.97k2025.03 | 72.1 | |
| SFTBase Model=Qwen2.5-Math-1.5B, Evaluation Protocol=Zero-shot2025.10 | 70.2 | |
| BaselineBase Model=Qwen2.5-Math-1.5B, Evaluation Protocol=Zero-shot2025.10 | 68.1 |