MATH
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MATH 500
84Pass@8 Rate
6
MATH 500
73.6Pass@1 Rate
6
MATH 500
98.8Accuracy
6
MATH-500 OOD (test)
67Score (Sav.)
6
Math500
88.89EigenWD M1
6
MATH-500
85.5Accuracy (MATH-500)
6
MATH 500
90Avg@1 Score
6
MATH Streaming Answers
0.115Brier Score
6
MATH
68.3Latency (s)
6
MATH
0Average Latency
6
MATH Streaming
1,407.4Tokens to Target Accuracy (MATH Streaming)
6
MATH
0.7Average Latency (s)
6
MATH
31.4Accuracy
6
MATH Tier 1
38.5Success Rate (SR)
6
Math held-out task instances (test)
20.4Accuracy
6
MATH
94.2Pass@32
6
MATH
77.66Average Success Rate (Avg@32)
6
MATH 500
85.5Accuracy
6
MATH
87.71Avg@32 Accuracy
6
MATH
0.33Inference Cost (USD)
6
MATH500
85.2P@1
6
MATH500
84.69Peak avg@32 Score
6
MATH500 in-distribution (test)
78.6Accuracy
6
MATH 500
79.7Mean Accuracy
6
MATH (test)
53.6Score
6