Mathematical Reasoning
Benchmarks
TaskDatasetSOTA resultTrendResults
Mathematical Reasoning Aggregate
75.4Average Score
46
Mathematical Reasoning Macro Average
69Pass@16 (Avg)
24
Mathematical Reasoning In-Distribution various (test)
66.4AIME 24 Score
22
Mathematical Reasoning Average
53.4Pass@1
18
Mathematical Reasoning Far-shift OOD
96.54AUROC
14
Mathematical Reasoning Near-shift OOD
98.76AUROC
14
Mathematical Reasoning AIME 24/25, AMC, MATH-500, Minerva, Olympiad
34.2AIME 2024 Score
13
Mathematical Reasoning benchmarks (AIME2024, AMC, MATH-500, Minerva, Olympiad)
33.5AIME 2024 Score
12
Mathematical Reasoning (MATH500, AIME25, OlympiadBench, AMC23) 2023/2025 (test)
86.2MATH Score
12
Mathematical Reasoning OOD (Near-shift)
0.159Kendall Tau
12
Mathematical Reasoning Far-shift OOD
0.161Kendall's Tau
12
Mathematical Reasoning Out-of-distribution
76.5MMLU-Pro
10
Mathematical Reasoning Datasets
128SFT Data Used
9
Mathematical Reasoning Aggregate
61.5Average Accuracy (Pass@1)
8
Mathematical Reasoning (GSM8K, MATH, STEM, TABMWP)
77.6GSM8K Accuracy
6
Mathematical Reasoning Average (AMC23, Minerva, Olympiad, Math500, AIME24, AIME25)
38.9Acc@16
6
Mathematical Reasoning Held-out (test)
48.92Accuracy
5
Mathematical Reasoning Perturbation Experiments
76.2Robustness Perturbation Success Rate (R-PSR)
4
Mathematical Reasoning (train)
1,177.8MTL (Loss)
4
Mathematical Reasoning (OOD)
89.9Algebra222 Accuracy
4
Mathematical Reasoning (in-distribution)
81.7GSM8K Score
4
Mathematical Reasoning Average
38.66Mean@8 Accuracy
3
Mathematical Reasoning
47.19CA
2
Mathematical reasoning datasets
—Primary metric
0