MATH
Benchmarks
TaskDatasetSOTA resultTrendResults
MATH500
97.9Pass@1 Accuracy
15
Math 500
83.6Accuracy
15
MATH
60.12Exact Match (%)
15
MATH 500
89.5Pass@1
15
MATH500
81.6Accuracy (@avg1)
15
MATH 500 Hard
80.8Accuracy
15
MATH-500
88.5Average@16
15
MATH standard (val)
31.33Accuracy
15
MATH 500
0.868Accuracy
15
MATH
89.8Success Score (MATH)
15
MATH 500
98.35P@1
15
MATH500
88.92Accuracy (k=8)
15
MATH Levels 3, 4, 5 (test)
90Overall Accuracy
15
MATH
84.84Exact Match
15
MATH 500
54.4Accuracy
14
MATH 500
9.64Speedup
14
MATH 500
77.6Accuracy (greedy pass@1)
14
MATH
13Normalized Spearman Distance
14
MATH 500
66.72Mean Accuracy (MATH 500)
14
MATH
66.69Accuracy (Mean)
14
MATH
79.7Accuracy
14
Math-500
93.28pass@3
14
MATH500
8.5Speedup
14
MATH 500
30.8Accuracy (MATH 500)
14
Math500 decontaminated
77.8Accuracy
14