Language Understanding on HellaSwag, PIQA, MMLU Suite
49HellaSwag AccuracyOpenLLaMA (V1)
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| OpenLLaMA (V1)Parameters=3B2023.12 | 49 | 75 | 26.95 | 49.9 | |
| INCITE (V1)Parameters=3B2023.12 | 48 | 74 | 26.75 | 48.48 | |
| MobileLLaMAParameters=2.7B2023.12 | 48 | 75 | 27.3 | 49.23 | |
| MobileLLaMAParameters=1.4B2023.12 | 43 | 73 | 24.97 | 44.9 | |
| OPTParameters=1.3B2023.12 | 41 | 71 | 24.61 | 44.66 | |
| TinyLLaMA (2T)Parameters=1.1B2023.12 | 40 | 70 | 25.41 | 44.84 | |
| PythiaParameters=1.4B2023.12 | 40 | 71 | 25.68 | 45.77 | |
| GalacticaParameters=1.3B2023.12 | 34 | 63 | 26.75 | 44.88 |