MATH
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Math Qwen2-7B-Instruct v1 (test)
3.26Acceptance Length (τ)
4
MATH (test)
24.9Final Test Accuracy
4
MATH
3.05MAT Score
4
MATH 500
88.8R0 Coverage
4
MATH 500
49.4MATH Accuracy (BoN@8)
4
MATH
40Accuracy
4
Math GSM8K, AQuA, AIME
92.4GSM8K Accuracy
4
MATH 500
95.4Accuracy (MATH 500)
4
MATH-500 stratified sample first 100 (test)
57Accuracy (%)
4
MATH (test)
40.9Mean Match Ratio
4
MATH
16.64Accuracy (MATH)
4
MATH-500 In-Distribution (test)
76.15CPT (90%)
4
MATH-500 OOD
63.55CPT (85%)
4
MATH 500
87.63CPT (95%)
4
MATH 500
74.94CPT (90%)
4
MATH 500 (ID)
53.79CPT (80%)
4
MATH-500 (ID queries)
64.97CPT (85%)
4
MATH 500
86.34CPT (95%)
4
MATH 500
0.939Hypervolume
4
MATH 500 OOD
53.58CPT (80%)
4
UC Berkeley MATH
1Success Rate (SR)
4
Math Specialized Target (test)
49.7Weighted Average Score
4
MATH500
45.01Avg@8 Score (MATH500)
4
MATH (test)
0.051Brier Score
4
MATH L4
49.31Mean@16
4