MATH
Benchmarks
TaskDatasetSOTA resultTrendResults
MATH500
22.2Accuracy
2
MATH (test)
25.34Pass@1
2
MATH500
94.5Accuracy (Avg@16)
2
MATH 500
89.8Accuracy
2
MATH 500
2,025,345Avg Token Usage (R0)
2
MATH
72.5NFE
2
MATH 500
75.93Avg Score @64
2
MATH 500
0.203Det.
2
Math Task
33Comprehensive Score
2
MATH 500
3.95Tau (Mean Accepted Tokens)
2
MATH 500
69.6Accuracy
2
MATH500
65.1MATH500 Score
2
MATH (test)
0.749AUROC
2
MATH 500
217.22Output Throughput
2
MATH-500
22.2Final Accuracy
2
MATH
9.8Final Accuracy
2
MATH500
0.668Avg@32
2
Math-500 4-shot (test)
37.83Throughput
2
MATH
83.8Pass@50
2
MATH
52.26Score
2
MATH
3.55Inference Cost (USD)
2
Math lm-evaluation-harness
1.44GSM8k Accuracy
2
MATH-500
77Accuracy
2
MATH500 EN
81.2Score
2
MATH500 dLLM-Var
39.8Accuracy
2