MATH
Benchmarks
TaskDatasetSOTA resultTrendResults
MATH (test)
74.2Pass@1
9
MATH
66.6Standard Correctness (SC)
9
MATH 4-shot
15.59Score
9
MATH 4-shot
5.27Accuracy
9
MATH-500
80Pass@1 Accuracy
9
MATH L5
0.35FDR
9
MATH500
2.5FDR
9
MATH 500 (test)
85.4Pass@1 Acc
9
MATH
67.75Pass@8
9
MATH 500
77.88Mean@8
9
MATH
49.9Solution Rate
9
MATH
0.916Score
9
MATH (test)
3.98Tokens Per Frame (TPF)
9
MATH500
80.4Avg@32
9
MATH 500
29.2Accuracy (128 tokens)
9
MATH500
91.2Avg@3 Accuracy
9
MATH 500
95.2Accuracy
9
Math
94.5Avg Performance
9
MATH500
94.8Pass@16
9
MATH500
93Pass@8
9
MATH
88.42Pass@1
9
MATH
38.4Accuracy @ t1
9
MATH 500
2.78Speedup
8
MATH500
56Mean Accuracy (MATH500)
8
MATH (test)
79.9Mean Hit Rate
8