MATH
Benchmarks
TaskDatasetSOTA resultTrendResults
MATH500
90Maj@8
8
MATH 500
4.53Latency (ms/tok)
8
MATH 500
5.17Latency (ms/tok)
8
MATH L1-L5
92.94Accuracy (L1)
8
Math (test)
73.53Accuracy
8
MATH 500
70Expert Count
8
MATH-500
66.6Avg@4 Score
8
MATH
76.8Exact Match
8
MATH 500
85.1Pass@1
8
Math OpenWebMath
2.87Speedup
8
Math HMMT (22 problems, 10 permutations)
398.9Average Cost
8
Math (Average of Math-500, GSM8K, and Big-Math)
72.9Accuracy
8
MATH 500
96.77Accuracy
8
Math Out-of-Domain
67.59Accuracy
8
MATH500
76.2MATH500 Score
8
MATH 500 samples
47.83Accuracy (MATH 500)
8
MATH
97Normalized Score
8
MATH
82.5AUROC
8
MATH 500
82.6Accuracy
8
MATH500
89.2Accuracy
8
MATH500
65.2ROC-AUC (Threshold 128)
8
MATH (test)
80.83Accuracy (MATH Test)
8
MATH 500
92.16Pass@1
8
MATH
38.4Accuracy
8
Math AMC12 + AIME24 25 + HMMTFeb2025
0.97AUROC
8