MATH
Benchmarks
TaskDatasetSOTA resultTrendResults
MATH (test)
46.38Accuracy
11
MATH 500
97.3Overall Accuracy
11
MATH lighteval
98.4Overall Accuracy
11
MATH session=7
58.75AA
11
Math Avg
47.6Accuracy
11
MATH 500
100Completion Rate
11
MATH (test)
7.26ECE
11
MATH
6.66ECE
11
MATH 500-sample (test)
89.2Accuracy (avg@1)
11
Math
42.2Mean Path Length (m)
11
MATH
59.87Solve Rate
11
MATH500
93.6Average Score (avg@1)
11
MATH-500
91.9AUCOAA
11
MATH500
80.6Accuracy
11
MATH
0.513CoT NLL
11
MATH-500
82.65Mean@4
11
MATH 500
40.6Accuracy
10
MATH 150 approach-feasible subset
2.302Cov@16
10
Math
22.71Overall Score
10
MATH X-Domain (ID)
83.4Accuracy
10
Minerva MATH
44.73Accuracy (Avg)
10
MATH500
92.07Average Accuracy (MATH500)
10
MATH 500
93Accuracy
10
MATH 500
89.38Average Score (k=8)
10
Math dataset
85.7Accuracy
10