Mathematical Problem Solving on TheoremQA TQ-Math
57.7Exact Match AccuracyDeepMath (RULES)
Evaluation results
| Method | Links | |
|---|---|---|
| DeepMath (RULES)Optimization=RULES2026.02 | 57.7 | |
| DeepMath (GRPO)Optimization=GRPO2026.02 | 56.1 | |
| DeepMath (vanilla)Optimization=vanilla2026.02 | 55.4 | |
| Qwen2.5 (RULES)Optimization=RULES2026.02 | 50.5 | |
| Qwen2.5 (GRPO)Optimization=GRPO2026.02 | 50 | |
| Qwen2.5 (vanilla)Optimization=vanilla2026.02 | 49.8 | |
| Llama-3.1 (vanilla)Optimization=vanilla2026.02 | 33.5 | |
| Llama-3.1 (RULES)Optimization=RULES2026.02 | 33 | |
| Llama-3.1 (GRPO)Optimization=GRPO2026.02 | 32.6 | |
| OLMO2 (GRPO)Optimization=GRPO2026.02 | 30.3 | |
| OLMO2 (vanilla)Optimization=vanilla2026.02 | 27.1 | |
| OLMO2 (RULES)Optimization=RULES2026.02 | 26.9 |