Mathematical Reasoning on DeepMath-103K (test)
43.27AccuracyLog- 1 / ln 202
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Log- 1 / ln 2022026.07 | 43.27 | 0.6119 | 0.1952 | 0.2375 | 0.0556 | 0.0527 | |
| Overconfidence-4k=42026.07 | 42.94 | 0.6609 | 0.2052 | 0.2242 | 0.0155 | -0.014 | |
| Correctness-only2026.07 | 42.38 | 0.5034 | -0.1437 | 0.5675 | 0.5686 | -0.5684 | |
| Overconfidence-1000k=10002026.07 | 42.04 | 0.5446 | 0.1737 | 0.2468 | 0.0822 | -0.0821 | |
| Underconfidence-1k=12026.07 | 41.59 | 0.6375 | 0.187 | 0.2289 | 0.0296 | -0.0153 | |
| Overconfidence-1k=12026.07 | 41.38 | 0.6752 | 0.193 | 0.2207 | 0.0341 | 0.0299 | |
| Brier-12026.07 | 40.63 | 0.6558 | 0.1828 | 0.2235 | 0.0468 | 0.0082 | |
| Underconfidence-4k=42026.07 | 39.21 | 0.6452 | 0.1684 | 0.2237 | 0.0422 | 0.036 | |
| Base (XML)Format=XML2026.07 | 32.28 | 0.5577 | -0.2743 | 0.5971 | 0.6145 | -0.6078 | |
| Brier-log Hybrid2026.07 | 26.35 | 0.6302 | 0.0781 | 0.1854 | 0.0273 | 0.0271 | |
| Base (JSON)Format=JSON2026.07 | 17.96 | 0.5713 | -0.5268 | 0.7063 | 0.7427 | -0.7415 | |
| SFT (XML)Format=XML2026.07 | 16.34 | 0.5732 | -0.5626 | 0.726 | 0.7636 | -0.7633 | |
| Brier-22026.07 | 0.02 | — | 0 | 0.0002 | 0.0048 | -0.0048 | |
| Log Loss2026.07 | 0.01 | — | 0 | 0.0002 | 0.0049 | -0.0049 | |
| Brier Score2026.07 | 0.01 | — | 0 | 0.0001 | 0.0049 | -0.0049 | |
| Log-12026.07 | 0 | — | 0 | 0.0001 | 0.005 | -0.005 |