Reading Comprehension on C3 (test)
77.38AccuracyQwen-1.5 14B (Teacher)
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen-1.5 14B (Teacher)Model Scale=14B2024.07 | 77.38 | |
| Qwen-1.5 1.8B + DDKDistillation=DDK2024.07 | 65.53 | |
| Qwen-1.5 1.8B + KDDistillation=KD2024.07 | 63.92 | |
| Qwen-1.5 1.8B + MiniLLMDistillation=MiniLLM2024.07 | 63.37 | |
| Qwen-1.5 1.8B + TEDDistillation=TED2024.07 | 63.18 | |
| Qwen-1.5 1.8B + CPT & DoReMiDistillation=CPT & DoReMi2024.07 | 60.85 | |
| Qwen-1.5 1.8B + CPTDistillation=CPT2024.07 | 60.3 | |
| Qwen-1.5 1.8B (Student)Model Scale=1.8B2024.07 | 58.27 | |
| LLaMA2-13BModel Size=13B, Shots=0-shot2024.07 | 46.03 | |
| TinyLLaMA-1.1B + DDKModel Size=1.1B, Shots=0-shot2024.07 | 42.41 | |
| TinyLLaMA-1.1B + CPT & DoReMiModel Size=1.1B, Shots=0-shot2024.07 | 39.02 | |
| TinyLLaMA-1.1B + CPTModel Size=1.1B, Shots=0-shot2024.07 | 38.91 | |
| TinyLLaMA-1.1B + TEDModel Size=1.1B, Shots=0-shot2024.07 | 37.61 | |
| TinyLLaMA-1.1B + KDModel Size=1.1B, Shots=0-shot2024.07 | 37.43 | |
| TinyLLaMA-1.1B + MiniLLMModel Size=1.1B, Shots=0-shot2024.07 | 37.18 | |
| TinyLLaMA-1.1BModel Size=1.1B, Shots=0-shot2024.07 | 35.24 |