MATH
Benchmarks
TaskDatasetSOTA resultTrendResults
MATH
0.32Cost ($)
3
LightEval MATH-Hard Level 5 only (val)
66Accuracy
3
Math500
0.846AVG@8 Score
3
MATH-500 (In-distribution)
77Acc
3
MATH500
0.691Pearson r
3
MATH-500 OpenR1 Harder
96.6Accuracy
3
MATH 500
42.4Accuracy
3
MATH 8-shot
73.89Accuracy
3
Math
237.96Throughput (tokens/s)
3
MATH Full (test)
0.539Overall Accuracy
3
MATH-500 (unsat.)
77.3Best Single Router Performance
2
MATH500
3.116Accuracy (Prediction of D'_pre)
2
MATH
0.48Correlation Coefficient (s)
2
MATH500
100Optimization Steps
2
Math500
94.12Pass@2
2
MATH
5Positive Seeds
2
Math
29.94Accuracy
2
MATH 500
69.2Strict-scorer Accuracy
2
Math fuzzed replay
0FPR
2
MATH
58Accuracy
2
MATH-500 Level 5
47.2Error Rate
2
MATH500
50Accuracy
2
MATH-500
66.2Score
2
MATH 500
97.2Score
2
Math avg. of MATH500, AIME25, Minerva Math, and Olympiad Bench
2.93Rollout Speedup Factor
2