MATH
Benchmarks
TaskDatasetSOTA resultTrendResults
MATH (test)
96Accuracy
59
MATH (val)
41.6Accuracy
59
MATH500 1.0 (test)
96.9Accuracy
57
MATH500
92.1Accuracy
57
MATH
61.68Average Accuracy (AA)
56
Math500
93Accuracy (%)
56
MATH
92Accuracy
55
MATH
92.8Accuracy
55
MATH 500
0.992Mean@1
55
MATH 500
0.073Brier Score
54
MATH 500
97Accuracy (%)
54
MATH 500
7.99Speedup
52
Math
80.8Accuracy (%)
52
MATH
83.48AUC
51
Math10K
78.2SVAMP
51
Math500
94.81Accuracy
50
MATH
96.7Score
50
MATH 500
40.2Mean Wall-Clock Time
49
MATH-500
66.5Accuracy
48
MATH-500
84.1Token Savings
48
MATH OOD
80.2Accuracy
48
MATH
38.36Accuracy
48
MATH500
94Accuracy
46
MATH
80.6Accuracy
46
MATH
0.85AUROC
46