Mathematical Reasoning on AMC23 (Pass@k)
98.6Pass@kQwen2.5-7B + PPO
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen2.5-7B + PPOk=128, Base Model=Qwen2.5-7B, RL Method=PPO2025.09 | 98.6 | |
| Qwen2.5-7Bk=128, Base Model=Qwen2.5-7B2025.09 | 98.4 | |
| Qwen2.5-7B + GRPO w/ VERL.k=128, Base Model=Qwen2.5-7B, RL Method=GRPO, Framework=VERL2025.09 | 98.3 | |
| Qwen2.5-7B + PPO w/ VERL.k=128, Base Model=Qwen2.5-7B, RL Method=PPO, Framework=VERL2025.09 | 98 | |
| Qwen2.5-7B + GRPOk=128, Base Model=Qwen2.5-7B, RL Method=GRPO2025.09 | 97.8 | |
| Mathstral-7B-v0.1 + GRPO w/ VERL.k=128, Base Model=Mathstral-7B-v0.1, RL Method=GRPO, Framework=VERL2025.09 | 97 | |
| Qwen2.5-3B + PPO w/ VERL.k=128, Base Model=Qwen2.5-3B, RL Method=PPO, Framework=VERL2025.09 | 96.8 | |
| Qwen2.5-3Bk=128, Base Model=Qwen2.5-3B2025.09 | 96.7 | |
| Qwen2.5-3B + PPOk=128, Base Model=Qwen2.5-3B, RL Method=PPO2025.09 | 96.5 | |
| Qwen2.5-3B + GRPO w/ VERL.k=128, Base Model=Qwen2.5-3B, RL Method=GRPO, Framework=VERL2025.09 | 95.9 | |
| Llama-3.2-3B-Instruct + GRPO w/ VERL.k=128, Base Model=Llama-3.2-3B-Instruct, RL Method=GRPO, Framework=VERL2025.09 | 95.7 | |
| Llama-3.2-3B-Instruct + GRPOk=128, Base Model=Llama-3.2-3B-Instruct, RL Method=GRPO2025.09 | 95.4 | |
| Llama-3.1-8B-Instruct + GRPO w/ VERL.k=128, Base Model=Llama-3.1-8B-Instruct, RL Method=GRPO, Framework=VERL2025.09 | 95.1 | |
| Llama-3.1-8B-Instruct + GRPOk=128, Base Model=Llama-3.1-8B-Instruct, RL Method=GRPO2025.09 | 94.9 | |
| Llama-3.2-3B-Instruct + PPO w/ VERL.k=128, Base Model=Llama-3.2-3B-Instruct, RL Method=PPO, Framework=VERL2025.09 | 94.7 | |
| Llama-3.1-8B-Instructk=128, Base Model=Llama-3.1-8B-Instruct2025.09 | 94.6 | |
| Llama-3.2-3B-Instruct + PPOk=128, Base Model=Llama-3.2-3B-Instruct, RL Method=PPO2025.09 | 94.5 | |
| Mathstral-7B-v0.1 + PPO w/ VERL.k=128, Base Model=Mathstral-7B-v0.1, RL Method=PPO, Framework=VERL2025.09 | 93.8 | |
| Llama-3.2-3B-Instructk=128, Base Model=Llama-3.2-3B-Instruct2025.09 | 93.5 | |
| Llama-3.1-8B-Instruct + PPOk=128, Base Model=Llama-3.1-8B-Instruct, RL Method=PPO2025.09 | 92.4 | |
| Qwen2.5-3B + GRPOk=128, Base Model=Qwen2.5-3B, RL Method=GRPO2025.09 | 92.2 | |
| Llama-3.1-8B-Instruct + PPO w/ VERL.k=128, Base Model=Llama-3.1-8B-Instruct, RL Method=PPO, Framework=VERL2025.09 | 91.9 | |
| Mathstral-7B-v0.1 + PPOk=128, Base Model=Mathstral-7B-v0.1, RL Method=PPO2025.09 | 91.7 | |
| Mathstral-7B-v0.1k=128, Base Model=Mathstral-7B-v0.12025.09 | 88.5 | |
| Mathstral-7B-v0.1 + GRPOk=128, Base Model=Mathstral-7B-v0.1, RL Method=GRPO2025.09 | 87.3 | |
| Mistral-7B-v0.3k=128, Base Model=Mistral-7B-v0.32025.09 | 73.5 | |
| Mistral-7B-v0.3 + GRPO w/ VERL.k=128, Base Model=Mistral-7B-v0.3, RL Method=GRPO, Framework=VERL2025.09 | 64.5 | |
| Mistral-7B-v0.3 + GRPOk=128, Base Model=Mistral-7B-v0.3, RL Method=GRPO2025.09 | 63.2 | |
| GRPO(λ)-0.99Backbone=Qwen3-1.7B2026.05 | 60 | |
| GRPOBackbone=Qwen3-1.7B2026.05 | 57.5 | |
| GRPO(λ)-0.9Backbone=Qwen3-1.7B2026.05 | 57.5 | |
| S-trace-0.9Backbone=Qwen3-1.7B2026.05 | 57.5 | |
| OPOBackbone=Qwen3-1.7B2026.05 | 55 | |
| Mistral-7B-v0.3 + PPO w/ VERL.k=128, Base Model=Mistral-7B-v0.3, RL Method=PPO, Framework=VERL2025.09 | 46.5 | |
| Mistral-7B-v0.3 + PPOk=128, Base Model=Mistral-7B-v0.3, RL Method=PPO2025.09 | 46.4 |