Science Question Answering on ARC Easy (0-shot and 32-shot)
87.58AccuracyQwen-1.5 14B
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Qwen-1.5 14BModel Type=Teacher2024.07 | 87.58 | — | — | |
| Qwen-1.5 7BRole=Teacher2024.07 | 85.78 | — | — | |
| DDKTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 4B2024.07 | 84.1 | — | — | |
| MiniLLMTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 4B2024.07 | 83.92 | — | — | |
| Mixtral 8x7BActive Params=13B2024.01 | 83.1 | — | — | |
| KDTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 4B2024.07 | 82.19 | — | — | |
| Mistral 7BActive Params=7B2024.01 | 80.5 | — | — | |
| CPTTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 4B2024.07 | 80.2 | — | — | |
| LLaMA 2 70BActive Params=70B2024.01 | 79.9 | — | — | |
| LLaMA 1 33BActive Params=33B2024.01 | 79.6 | — | — | |
| Qwen-1.5 4BModel Type=Student2024.07 | 79.3 | — | — | |
| LLaMA 2 13BActive Params=13B2024.01 | 75.2 | — | — | |
| DDKBase Model=Qwen-1.5 1.8B, Teacher=Qwen-1.5 7B2024.07 | 74.43 | — | — | |
| MiniLLMBase Model=Qwen-1.5 1.8B, Teacher=Qwen-1.5 7B2024.07 | 73.86 | — | — | |
| KDBase Model=Qwen-1.5 1.8B, Teacher=Qwen-1.5 7B2024.07 | 73.74 | — | — | |
| LLaMA2-13BModel Size=13B, Shots=0-shot2024.07 | 73.3 | — | — | |
| CPTBase Model=Qwen-1.5 1.8B, Teacher=Qwen-1.5 7B2024.07 | 72 | — | — | |
| Qwen-1.5 1.8BRole=Student2024.07 | 70.23 | — | — | |
| LLaMA 2 7BActive Params=7B2024.01 | 68.7 | — | — | |
| TinyLLaMA-1.1B + DDKModel Size=1.1B, Shots=0-shot2024.07 | 36.29 | — | — | |
| TinyLLaMA-1.1B + MiniLLMModel Size=1.1B, Shots=0-shot2024.07 | 29.46 | — | — | |
| TinyLLaMA-1.1B + KDModel Size=1.1B, Shots=0-shot2024.07 | 29.4 | — | — | |
| TinyLLaMA-1.1B + TEDModel Size=1.1B, Shots=0-shot2024.07 | 28.61 | — | — | |
| TinyLLaMA-1.1B + CPTModel Size=1.1B, Shots=0-shot2024.07 | 28.06 | — | — | |
| TinyLLaMA-1.1B + CPT & DoReMiModel Size=1.1B, Shots=0-shot2024.07 | 27.75 | — | — | |
| TinyLLaMA-1.1BModel Size=1.1B, Shots=0-shot2024.07 | 24.18 | — | — | |
| OPT 1.3BScale=1.3B2022.12 | — | 50.8 | 60.2 | |
| OPT 175BScale=175B2022.12 | — | 67 | 76.8 | |
| OPT 30BScale=30B2022.12 | — | 63.9 | 72.7 | |
| OPT-IML 1.3BScale=1.3B2022.12 | — | 53.7 | 59.9 | |
| OPT-IML 175BScale=175B2022.12 | — | 70.8 | 77.2 | |
| OPT-IML 30BScale=30B2022.12 | — | 64.9 | 72.1 |