MATH
Benchmarks
TaskDatasetSOTA resultTrendResults
MATH500
79.2Accuracy
12
MATH-plus
107.1Average Latency (ms/token)
12
MATH-plus
157.83p90 Latency (ms/token)
12
MATH 500
4.37Speedup
12
MATH 500
67.56Mean@16
12
MATH 500
84.35Avg@8 Accuracy
12
MATH 500
63.25Avg@8 Score
12
MATH 500
64.53Avg@8
12
MATH500
40.2Accuracy
12
MATH-500 in-distribution (test)
79.51Average@32
12
MATH 500
84.6EM (Accuracy)
12
MATH
0Average FPR
12
MATH500
74Accuracy
12
MATH
82Accuracy
12
MATH500 Standard (Full)
49.4Avg@16
12
MATH500
81.6Accuracy (%)
12
MATH 500
0.98Wall-clock Time (s)
12
MATH 500
82.77Accuracy
12
MATH
38.8Accuracy
12
MATH 500
85.8Pass@1
12
Math
66.84Performance Score
12
MATH500
74.2Accuracy (MATH500)
12
MATH
90.8Accuracy (%)
12
MATH
2.2Latency (s)
12
MATH
0.662WP-AUC
12