Downstream Benchmarks
Benchmarks
TaskDatasetSOTA resultTrendResults
Zero-shot Downstream Benchmarks (BoolQ, ARC-E, ARC-C, HellaSwag)
83.2BoolQ Accuracy
18
Downstream Benchmarks (HellaSwag, SIQA, PIQA, OBQA, WinoGrande, RACE)
42.96HellaSwag
10
Downstream Benchmarks (Winogrande, HellaSwag, ARC-E, ARC-C, MMLU) 0.3B-token distillation
82.6Winogrande Accuracy
8
9 Downstream Benchmarks Average
0.456Average Accuracy
6
Downstream Benchmarks Zero-shot (Hellaswag, ARC, PIQA, BoolQ, SciQ)
29.23Hellaswag Accuracy (Zero-shot)
4