Mathematical Reasoning on DeepMath (test)
62Pass@1GRPO
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GRPOBackbone=Qwen3-4B-Instruct2026.02 | 62 | 188.35 | 7.09 | — | — | — | |
| DRGRPOBackbone=Qwen3-4B-Instruct2026.02 | 61.2 | 185.17 | 7.29 | — | — | — | |
| GRPOBackbone=Qwen2.5-7B-Instruct2026.02 | 53.5 | 8.15 | 1 | — | — | — | |
| WS-GRPOBackbone=Qwen3-4B-Instruct2026.02 | 53.2 | 16.29 | 2 | — | — | — | |
| WS-GRPOBackbone=Qwen2.5-7B-Instruct2026.02 | 49.4 | 16.18 | 2 | — | — | — | |
| DRGRPOBackbone=Qwen2.5-7B-Instruct2026.02 | 48.4 | 8.34 | 1 | — | — | — | |
| WS-GRPOBackbone=Llama-3.2-3B-Instruct2026.02 | 46.4 | 12.66 | 1 | — | — | — | |
| GRPOBackbone=Llama-3.1-8B-Instruct2026.02 | 46.4 | 9.09 | 1 | — | — | — | |
| DRGRPOBackbone=Llama-3.1-8B-Instruct2026.02 | 46.3 | 512 | 1 | — | — | — | |
| DRGRPOBackbone=Llama-3.2-3B-Instruct2026.02 | 46 | 8.14 | 1.01 | — | — | — | |
| GRPOBackbone=Llama-3.2-3B-Instruct2026.02 | 45.5 | 9.02 | 1 | — | — | — | |
| WS-GRPOBackbone=Llama-3.1-8B-Instruct2026.02 | 37.2 | 41.49 | 2 | — | — | — | |
| DAPOModel Scale=1.5B, Budget Type=ample, Rollout count per prompt (n)=128, Target Accuracy=44.0%2026.05 | — | — | — | 42.02 | — | — | |
| DAPOModel Scale=1.5B, Budget Type=limited, Rollout count per prompt (n)=16, Target Accuracy=41.3%2026.05 | — | — | — | 4.49 | — | — | |
| GRPOModel Scale=1.5B, Budget Type=ample, Rollout count per prompt (n)=128, Target Accuracy=44.0%2026.05 | — | — | — | 15.97 | — | — | |
| GRPOModel Scale=1.5B, Budget Type=limited, Rollout count per prompt (n)=16, Target Accuracy=41.3%2026.05 | — | — | — | 1.97 | — | — | |
| Pilot-CommitModel Scale=1.5B, Budget Type=ample, Rollout count per prompt (n)=128, Target Accuracy=44.0%2026.05 | — | — | — | 10.57 | 1.5 | 4 | |
| Pilot-CommitModel Scale=1.5B, Budget Type=limited, Rollout count per prompt (n)=16, Target Accuracy=41.3%2026.05 | — | — | — | 1.93 | 1 | 2.3 |