Common Sense Reasoning on ARC, BoolQ, RTE, Winogrande, and TruthfulQA
34ARC Challenge AccuracyOpenLLaMA (V1)
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| OpenLLaMA (V1)Parameters=3B2023.12 | 34 | 69 | 68 | 58 | 62 | 22 | 35 | |
| INCITE (V1)Parameters=3B2023.12 | 32 | 68 | 67 | 52 | 63 | 21 | 33 | |
| MobileLLaMAParameters=2.7B2023.12 | 32 | 68 | 61 | 59 | 63 | 23 | 36 | |
| GalacticaParameters=1.3B2023.12 | 28 | 62 | 62 | 52 | 55 | 25 | 41 | |
| PythiaParameters=1.4B2023.12 | 26 | 61 | 63 | 52 | 57 | 23 | 39 | |
| MobileLLaMAParameters=1.4B2023.12 | 26 | 61 | 53 | 53 | 59 | 21 | 35 | |
| OPTParameters=1.3B2023.12 | 24 | 57 | 56 | 51 | 59 | 24 | 39 | |
| TinyLLaMA (2T)Parameters=1.1B2023.12 | 23 | 57 | 59 | 55 | 57 | 23 | 39 |