MATH
Benchmarks
TaskDatasetSOTA resultTrendResults
MATH replicated controlled leaks in OpenWebText
74.2Recall
3
MATH500 Long context
89.12Accuracy
3
MATH500
66.92Mean@8 Accuracy
3
MATH 500
89Mean@16 Accuracy
3
MATH-500 Level 5
26.1AUROC
3
MATH
61.78Teacher Accuracy
3
MATH-500
34.4Accuracy
3
MATH 500
98.6Accuracy
3
MATH500
49Performance
3
MATH 500
90.49pass@16 Success Rate
3
MATH 500
48.4Accuracy (MATH 500)
3
MATH
50.81TPS
3
MATH
48.6Overall Accuracy
3
MATH Level 5
70Exact Match (EM)
3
Math
93.7Accuracy
3
MATH
40Accuracy
3
MATH-500 2023
13.6Peak Accuracy
3
MATH Num. Thy All Non-Easy
43.7Recovery Rate
3
MATH Num. Thy Med. + Hard
72.2Recovery Rate
3
MATH Geometry All Non-Easy
39.6Recovery Rate
3
MATH Geometry Med. + Hard
37.3Recovery Rate
3
MATH Algebra All Non-Easy
40Recovery Rate
3
MATH Algebra (Med. + Hard)
31Recovery Rate
3
MATH (test)
103.63TPS
3
MATH
13.2ECE
3