General Language Understanding on Open LLM Leaderboard HuggingFace 2023a (test)
59.4ARC-c Accuracy (25-shot)LLaMA-2-13B (Zero-shot)
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LLaMA-2-13B (Zero-shot)Pre-trained=LLaMA-2-13B, Tuning=None, Full-param=-2023.10 | 59.4 | 82.1 | 55.8 | 37.4 | 58.7 | |
| LoRAPre-trained=LLaMA-2-13B, Tuning=LoRA, Full-param=×2023.10 | 57.3 | 81.2 | 55.6 | 44.7 | 59.7 | |
| FT-AdamPre-trained=LLaMA-2-13B, Tuning=FT-Adam, Full-param=✓2023.10 | 57 | 81.2 | 55.8 | 50.9 | 61.2 | |
| FT-LionPre-trained=LLaMA-2-13B, Tuning=FT-Lion, Full-param=✓2023.10 | 56.6 | 81.2 | 55.6 | 50.7 | 61 | |
| QFTPre-trained=LLaMA-2-13B, Tuning=QFT, Full-param=✓2023.10 | 56.2 | 81 | 55.9 | 48.6 | 60.4 | |
| FT-BnbPre-trained=LLaMA-2-13B, Tuning=FT-Bnb, Full-param=✓2023.10 | 56 | 80.9 | 55.4 | 49.4 | 60.4 | |
| FT-AdamPre-trained=LLaMA-2-7B, Tuning=FT-Adam, Full-param=✓2023.10 | 53.6 | 77.3 | 49.4 | 51.5 | 58 | |
| FT-LionPre-trained=LLaMA-2-7B, Tuning=FT-Lion, Full-param=✓2023.10 | 53.5 | 77.6 | 49.2 | 51.3 | 57.9 | |
| LLaMA-2-7B (Zero-shot)Pre-trained=LLaMA-2-7B, Tuning=None, Full-param=-2023.10 | 53.1 | 78.6 | 46.9 | 38.8 | 54.4 | |
| FT-BnbPre-trained=LLaMA-2-7B, Tuning=FT-Bnb, Full-param=✓2023.10 | 53.1 | 76.9 | 49 | 51.1 | 57.5 | |
| LoRAPre-trained=LLaMA-2-7B, Tuning=LoRA, Full-param=×2023.10 | 53 | 78 | 47.8 | 45.8 | 56.2 | |
| QFTPre-trained=LLaMA-2-7B, Tuning=QFT, Full-param=✓2023.10 | 52.9 | 76.7 | 48.8 | 51.1 | 57.4 |