MATH
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MATH L2
64.31Mean@16
4
MATH L1
62.35Mean@16
4
MATH500 Llama3-8B (first 100 examples)
39Accuracy
4
MATH
1.22Training Cost (USD)
4
MATH (test)
75.3Pass@1
4
Math-500
9.55Throughput (tokens/s)
4
MATH
90pass@50
4
MATH
2.34Training Cost (USD)
4
MATH 500
59.7Mean Accuracy @16
4
MATH500
74.4Accuracy
4
MATH500 (val)
0.47Spearman Correlation
4
MATH-500
4.64Speedup
4
Math 500
76Score
4
Math (test)
75.82Accuracy
4
Math (test)
66.8AUROC
3
MATH
37.32Exact Match (EM)
3
MATH
55.1Pass@1
3
MATH-plus GPT-4
0.71Kendall's Tau
3
MATH
64.5Exact Match (EM)
3
MATH (retain)
38.5Accuracy (Retain)
3
Math Synth
56R%
3
Math XLow Tier
62.3R Score
3
Math Low Tier
62.8R Score
3
Math High Tier
65.3R (%)
3
Math Synthetic (test)
0.813AUROC
3