MATH
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MATH 500
95Pass@1
33
MATH 500
3.14Speedup
32
MATH
15.1MATH Score
32
MATH 2021 (test)
78.3pass@1
32
Math Benchmarks
76.9Accuracy (%)
32
Math
87F1 Score
32
MATH500
45.4Accuracy
32
MATH (test)
94.16Accuracy
32
MATH
89.4Pass@k (PK)
31
MATH-500
94.61Avg@16
31
MATH 500
10,943Throughput (tok/s)
30
MATH
87.6Ac (alpha=0.10)
30
MATH
62Removal Rate
30
Math
7.69Match Rate
30
MATH500 4-shot
42.2Accuracy
30
MATH500
91.4Pass@k
30
MATH Target domain
26.2Accuracy
30
MATH 500
83.1Avg@4
30
MATH500
83.8Acc
30
MATH 500
75.8Accuracy
30
MATH 500
97.3During-task Accuracy (MATH 500)
29
MATH lighteval
98.4During-task Accuracy
29
MATH500
6.76Speedup
28
Math tasks
81.6Easy Accuracy
28
Math Math500 Minerva
80.9Math500 Score
28