Mathematical Reasoning on DeepMath-103K (test) benchmark leaderboard · SOTA2 Research

Mathematical Reasoning on DeepMath-103K (test)

43.27AccuracyLog- 1 / ln 202

Evaluation Results

Method	Links
Log- 1 / ln 2022026.07		43.27	0.6119	0.1952	0.2375	0.0556	0.0527
Overconfidence-42026.07		42.94	0.6609	0.2052	0.2242	0.0155	-0.014
Correctness-only2026.07		42.38	0.5034	-0.1437	0.5675	0.5686	-0.5684
Overconfidence-10002026.07		42.04	0.5446	0.1737	0.2468	0.0822	-0.0821
Underconfidence-12026.07		41.59	0.6375	0.187	0.2289	0.0296	-0.0153
Overconfidence-12026.07		41.38	0.6752	0.193	0.2207	0.0341	0.0299
Brier-12026.07		40.63	0.6558	0.1828	0.2235	0.0468	0.0082
Underconfidence-42026.07		39.21	0.6452	0.1684	0.2237	0.0422	0.036
Base (XML)2026.07		32.28	0.5577	-0.2743	0.5971	0.6145	-0.6078
Brier-log Hybrid2026.07		26.35	0.6302	0.0781	0.1854	0.0273	0.0271
Base (JSON)2026.07		17.96	0.5713	-0.5268	0.7063	0.7427	-0.7415
SFT (XML)2026.07		16.34	0.5732	-0.5626	0.726	0.7636	-0.7633
Brier-22026.07		0.02	—	0	0.0002	0.0048	-0.0048
Log Loss2026.07		0.01	—	0	0.0002	0.0049	-0.0049
Brier Score2026.07		0.01	—	0	0.0001	0.0049	-0.0049
Log-12026.07		0	—	0	0.0001	0.005	-0.005