MATH
Benchmarks
TaskDatasetSOTA resultTrendResults
Math500
59.39Pass@4 Accuracy
2
MATH500
81.74Avg@16
2
MATH 500
90.4Accuracy
2
MATH500
90.4Accuracy
2
MATH-500
93.5Algebra Accuracy
1
MATH Cumulative (4 categories) (test)
2,747N
1
MATH Precalculus (test)
546N
1
MATH Counting & Probability (test)
474N
1
MATH Number Theory (test)
540N
1
MATH-500 Hard Level 4-5
85.9Score (SC)
1
MATH-500 (main)
91SC
1
MATH-500 Level 4
59.4Error Rate
1
MATH-500 Level 2
45.8Error Rate
1
MATH-500 Level 1
40.1Error Rate
1
MATH-500 Level 1
0.074Brier Score
1
MATH-500
71.98Pass@1
1
Math Hard
0.003Baseline Score
1
Math Medium
0.006Baseline Score
1
Math Easy
2.56Baseline Score
1
MATH Level 1–3
70Monotonicity Rate
1
MATH
49HALT Wins
1
Math 1000 samples (test)
75Accuracy
1
MATH
—Primary metric
0
MATH 500
—Mean Accuracy
0
Math1-3
—Primary metric
0