Instruction Following on DollyEval 500 (test)
31.3Rouge-LTeacher
Evaluation results
| Method | Links | ||
|---|---|---|---|
| TeacherTeacher-Student Configuration=LLaMA 13B → LLaMA 7B2026.06 | 31.3 | 0.891 | |
| ARKDTeacher-Student Configuration=LLaMA 13B → LLaMA 7B2026.06 | 29.1 | 0.868 | |
| MiniLLMTeacher-Student Configuration=LLaMA 13B → LLaMA 7B2026.06 | 29 | — | |
| FKL+RKLTeacher-Student Configuration=LLaMA 13B → LLaMA 7B2026.06 | 28.3 | 0.848 | |
| RKLTeacher-Student Configuration=LLaMA 13B → LLaMA 7B2026.06 | 28.2 | 0.844 | |
| TeacherTeacher-Student Configuration=GPT2 1.5B → GPT2 120M2026.06 | 27.1 | 0.842 | |
| TeacherTeacher-Student Configuration=GPT2 1.5B → GPT2 340M2026.06 | 27.1 | 0.858 | |
| FKLTeacher-Student Configuration=LLaMA 13B → LLaMA 7B2026.06 | 27.1 | 0.825 | |
| SeqKDTeacher-Student Configuration=LLaMA 13B → LLaMA 7B2026.06 | 26.6 | 0.818 | |
| ARKDTeacher-Student Configuration=GPT2 1.5B → GPT2 340M2026.06 | 26.1 | 0.827 | |
| SFT w/o KDTeacher-Student Configuration=LLaMA 13B → LLaMA 7B2026.06 | 26 | 0.81 | |
| FKL+RKLTeacher-Student Configuration=GPT2 1.5B → GPT2 340M2026.06 | 25.7 | 0.817 | |
| RKLTeacher-Student Configuration=GPT2 1.5B → GPT2 340M2026.06 | 25.5 | 0.813 | |
| MiniLLMTeacher-Student Configuration=GPT2 1.5B → GPT2 340M2026.06 | 25.4 | — | |
| FKLTeacher-Student Configuration=GPT2 1.5B → GPT2 340M2026.06 | 24.9 | 0.803 | |
| MiniLLMTeacher-Student Configuration=GPT2 1.5B → GPT2 120M2026.06 | 24.6 | — | |
| ARKDTeacher-Student Configuration=GPT2 1.5B → GPT2 120M2026.06 | 24.5 | 0.815 | |
| SeqKDTeacher-Student Configuration=GPT2 1.5B → GPT2 340M2026.06 | 24.2 | 0.798 | |
| SFT w/o KDTeacher-Student Configuration=GPT2 1.5B → GPT2 340M2026.06 | 24 | 0.793 | |
| RKLTeacher-Student Configuration=GPT2 1.5B → GPT2 120M2026.06 | 23.3 | 0.797 | |
| FKL+RKLTeacher-Student Configuration=GPT2 1.5B → GPT2 120M2026.06 | 23.2 | 0.794 | |
| FKLTeacher-Student Configuration=GPT2 1.5B → GPT2 120M2026.06 | 22.9 | 0.781 | |
| SFT w/o KDTeacher-Student Configuration=GPT2 1.5B → GPT2 120M2026.06 | 22.3 | 0.772 | |
| SeqKDTeacher-Student Configuration=GPT2 1.5B → GPT2 120M2026.06 | 21.9 | 0.765 |