MATH
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MATH (OOD)
71KL Div (alpha=1)
5
MATH 500
74.7Pass@1
5
Math (test)
30.1Accuracy
5
MATH In-Distribution (test)
56.1Final Answer Accuracy
5
MATH500
93.43Peak Avg Score
5
MATH 4-shot
800TPF
5
MATH500
88.4Accuracy
5
MATH 500
791.23Throughput (TPS)
5
285 models on one Math benchmark
100Top-10 Recall
5
Math Find
33.14Accuracy
5
MATH
79.5Accuracy
5
Math Gender (test)
36Accuracy
5
Math Race (test)
60Accuracy
5
MATH
14.1Pass@1
5
Math Base-11
33Accuracy (P_clean, Avg)
5
MATH (test)
24.68Exact Match (EM)
4
MATH 500
403.45TPS
4
MATH-plus R1
0.73Kendall's Tau
4
MATH
74.8Utility
4
MATH
57.41Vanilla Accuracy
4
MATH 500 LLaMA 3.1-8B-Instruct (test)
49.8Accuracy
4
MATH (test)
435p@6=0
4
MATH (retain)
37Accuracy
4
Math
41.93Utility Score
4
Math500
57.1Top-1 Accuracy
4