Reasoning Tasks
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Reasoning Tasks (BoolQ, PIQA, HellaSwag, WinoGrande, ARC-e, ARC-c, OBQA) Zero-shot
82.813BoolQ Accuracy (Zero-shot)
64
Zero-Shot Reasoning Tasks (ARC-C, ARC-E, BoolQ, Hella, OBQA, PIQA, SIQA, Wino)
65.53ARC-C Accuracy
54
Reasoning Tasks Average
68.6Average Score
32
Zero-shot Reasoning Tasks (ARC-e, ARC-c, BoolQ, Hellaswag, WinoGrande, MMLU)
77.1ARC-e Accuracy (Zero-shot)
20
Reasoning Tasks AIME24, AIME25, GPQA
92.2AIME 2024 Accuracy
18
Reasoning Tasks ARC-C, BoolQ, HellaSwag, MathQA, MMLU, Race
50.43ARC-C Accuracy
14
Reasoning Tasks Zero-shot (PI, AE, AC, HS, WG, LB)
81.6PI Score (Zero-shot)
13
Reasoning tasks
70.7Reasoning Accuracy
7
Reasoning Tasks Aggregate
0.81Spearman Rho
6
Reasoning Tasks
47Query Count
3