Reasoning Benchmarks
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Reasoning Benchmarks BBH, MMLU, ARC-C, ThmQA (test)
64.66BBH
66
Reasoning Benchmarks ARC-e, ARC-c, BoolQ, PIQA, SIQA, HellaS., OBQA, Wino.
72.6ARC-e Accuracy
38
15 reasoning benchmarks weighted mean (test)
81.35Accuracy
36
Reasoning Benchmarks Average
75.33Average Accuracy
34
Reasoning Benchmarks GSM8K, MATH-500, AIME24, AIME25, GPQA-D
95.15GSM8K Accuracy
33
Reasoning Benchmarks Zero-shot (PIQA, ARC, HellaSwag, WinoGrande)
82.75PIQA Accuracy
31
Reasoning Benchmarks Zero-shot
69.99Overall Zero-Shot Accuracy
26
Reasoning Benchmarks Aggregate
70.63Average Score
24
Reasoning Benchmarks Average
0.874Train R^2
21
Reasoning Benchmarks GPQA-Diamond AIME2024 MATH500 HumanEval
85.77Average Score
21
17 Reasoning Benchmarks Aggregate (test)
90.71Accuracy
21
Reasoning Benchmarks Zero-shot (BoolQ, PIQA, HellaSwag, WinoGrande, ARC)
71.1BoolQ Accuracy (Zero-shot)
20
Reasoning Benchmarks Average
81.28Pass@1
16
Reasoning Benchmarks Overall
5.81Delta Accuracy
16
Reasoning Benchmarks MATH, GSM8K, AQUA, GSM-H, MMLU, MMLU-P, GPQA, AIME
84.4MATH Accuracy
14
Reasoning Benchmarks Average
44.7Average Accuracy
12
Reasoning Benchmarks Average of MATH-500, AIME 24, AIME 25, GPQA Diamond, CommonsenseQA, LiveCodeBench, and LongBenchv2 Qwen3
74.8Accuracy
12
Reasoning Benchmarks Biology, Chemistry, Material, Physics, Tool Use
65.8Biology Accuracy
11
Reasoning Benchmarks (GSM8K, Math, AIME, HumanEval, LiveCodeBench)
85.12GSM8K Accuracy
9
Reasoning Benchmarks (GSM8K, Math, AIME, HumanEval, LiveCodeBench) (test)
87.23GSM8K Accuracy
9
Reasoning Benchmarks Zero-shot (ARC-e, ARC-c, BoolQ, PIQA, SIQA, HellaSwag, OBQA, WinoGrande)
74.2ARC-e Accuracy
8
Reasoning Benchmarks Zero-shot (ARC-C, ARC-E, HellaSwag, LAMBADA, OpenBookQA, PIQA, WinoGrande)
36.9ARC-C Accuracy
6
8 reasoning benchmarks (including GSM8K, MATH500, AIME 24, AIME 25, and OlympiadBench) (test)
53.3Token Savings
5
Reasoning Benchmarks (AIME24, AIME25, AMC, MATH, Minerva, Olympiad) (test)
28.33AIME24 Accuracy
4
Reasoning Benchmarks ARC-C, ARC-E, BoolQ, HellaSwag, PIQA, WinoGrande
60.15ARC-C Accuracy
3