Lemma Judging on NLPS (test)
86.5Exact Match AccuracyLlama-3.1 (GRPO)
Evaluation results
| Method | Links | |
|---|---|---|
| Llama-3.1 (GRPO)Optimization=GRPO2026.02 | 86.5 | |
| Llama-3.1 (RULES)Optimization=RULES2026.02 | 86.4 | |
| DeepMath (RULES)Optimization=RULES2026.02 | 86.2 | |
| Qwen2.5 (RULES)Optimization=RULES2026.02 | 85.8 | |
| OLMO2 (GRPO)Optimization=GRPO2026.02 | 84.2 | |
| Qwen2.5 (GRPO)Optimization=GRPO2026.02 | 83.7 | |
| OLMO2 (RULES)Optimization=RULES2026.02 | 83.4 | |
| DeepMath (GRPO)Optimization=GRPO2026.02 | 77.2 | |
| Llama-3.1 (vanilla)Optimization=vanilla2026.02 | 75.1 | |
| Qwen2.5 (vanilla)Optimization=vanilla2026.02 | 71.9 | |
| OLMO2 (vanilla)Optimization=vanilla2026.02 | 69.1 | |
| DeepMath (vanilla)Optimization=vanilla2026.02 | 51.5 |