Equation Number Guessing on The benchmark (held-out slice)
59AccuracyPython Solver Oracle
Evaluation Results
| Method | Links | |
|---|---|---|
| Python Solver Oracletype=backtracking / generate-and-verify2026.06 | 59 | |
| Python Solver Oracletype=reverse-engineered Python program2026.06 | 20.6 | |
| Solver-Grounded Synthetic CoT (Model)adapter=fine-tuned rank-≤32, decoding=greedy CoT2026.06 | 18 | |
| Base Model2026.06 | 3.7 | |
| Solver-Grounded Synthetic CoT (Model)adapter=fine-tuned rank-≤32, decoding=greedy CoT2026.06 | 2 | |
| Base Model2026.06 | 0 |