Zero-shot Reasoning on (ARC-E, PIQA, SciQ, HellaSwag, LAMBADA, WinoGrande, BoolQ)
51.22ARC-EInheritune
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| InherituneModel Layers=24, Training Strategy=Inheritune, Pre-training Data=OpenWebText2024.04 | 51.22 | 66.87 | 79.2 | 34.2 | 43.3 | 53.28 | 60.4 | 55.5 | |
| GPT-2 XLarge (Full model)Model Layers=48, Training Strategy=from scratch, Pre-training Data=OpenWebText, Parameters=1.5B2024.04 | 50.38 | 66.7 | 77 | 33.65 | 39.9 | 51.93 | 57.86 | 53.92 |