Downstream Tasks
Benchmarks
TaskDatasetSOTA resultTrendResults
Downstream Tasks Zero-shot
76Accuracy
278
Downstream Tasks MMLU, PIQA, Arc-E, Arc-C, Wino, OpenQA
77.62MMLU
218
Downstream Tasks Zero-shot (BoolQ, HellaSwag, WinoGrande, ARC-e, ARC-c, PIQA, OBQA)
81.65BoolQ Accuracy
87
6 zero-shot downstream tasks
80.05Average Accuracy
70
Zero-shot Downstream Tasks (ARC, HellaSwag, WinoGrande, BoolQ, PiQA)
81Average Accuracy (Zero-Shot)
48
Downstream Tasks (test)
73.4Accuracy
36
Downstream Tasks Summary
61.5Average Accuracy
36
11 Downstream Tasks Aggregate
64.6Average Accuracy
32
Eight downstream tasks zero-shot
47.7Accuracy (Zero-shot)
30
Downstream Tasks PiQA ARC Hellaswag Winogrande BoolQ
84.4PiQA Accuracy (Zero-shot)
30
7 Downstream Tasks Avg
74.1Average Score
28
Downstream tasks Average (test)
58.02Average Score
28
Downstream Tasks (LMB, PIQA, HellaSwag, OPQA, ARC)
32.75LAMBADA (LMB) Accuracy
22
62 downstream tasks
67.5Average Accuracy
18
Downstream tasks average
81Avg Zero-shot Accuracy
16
6 downstream tasks (ARC-Challenge, ARC-Easy, HellaSwag, Winogrande, SciQ, PIQA)
43.6ARC-Challenge Accuracy
12
Downstream Tasks
63.4Average Accuracy
12
Downstream Tasks Aggregate
60.49Accuracy
11
Downstream Tasks (PiQA, ARC-E, ARC-C, HellaSwag, Winogrande, BoolQ, OBQA, SiQA) Zero-shot
80.63PiQA Accuracy
10
9 Downstream Tasks Utility
54.7Average Accuracy
10
10 Downstream Tasks
52.79Average Accuracy
9
Downstream Tasks PiQA, Arc E, Arc C, HS, WG, BoolQ
80.5PiQA Accuracy
9
Downstream Tasks (ARC-C, ARC-E, H'SWAG, LAM'DA, OPENBKQA, PIQA, W'GRANDE) zero-shot
32.76ARC-C Score
9
Downstream Tasks (ARC, HellaSwag, PIQA, Winogrande) Standard LLM Eval (val/test)
56.98Average Accuracy
9
Downstream Tasks (ARC-E, ARC-C, SciQ, PIQA, MMLU, CMMLU, CEVAL)
59.21Average Accuracy
9