Mathematical Problem Solving on Putnam-Axiom (PA)
39.5Exact Match AccDeepMath (RULES)
Evaluation results
| Method | Links | |
|---|---|---|
| DeepMath (RULES)Optimization=RULES2026.02 | 39.5 | |
| DeepMath (vanilla)Optimization=vanilla2026.02 | 39.1 | |
| DeepMath (GRPO)Optimization=GRPO2026.02 | 37.9 | |
| Qwen2.5 (GRPO)Optimization=GRPO2026.02 | 35.6 | |
| Qwen2.5 (vanilla)Optimization=vanilla2026.02 | 32.4 | |
| Qwen2.5 (RULES)Optimization=RULES2026.02 | 32 | |
| Llama-3.1 (GRPO)Optimization=GRPO2026.02 | 13.2 | |
| Llama-3.1 (RULES)Optimization=RULES2026.02 | 12.8 | |
| Llama-3.1 (vanilla)Optimization=vanilla2026.02 | 11.5 | |
| OLMO2 (vanilla)Optimization=vanilla2026.02 | 8.4 | |
| OLMO2 (GRPO)Optimization=GRPO2026.02 | 7.8 | |
| OLMO2 (RULES)Optimization=RULES2026.02 | 6.7 |