Instruction Following on VicunaEval (OOD)
19.7Rouge-LTeacher
Evaluation Results
| Method | Links | |
|---|---|---|
| TeacherBackbone=LLaMA, Parameters=13B2026.06 | 19.7 | |
| ARKDBackbone=LLaMA, Parameters=7B2026.06 | 19.2 | |
| FKL+RKLBackbone=LLaMA, Parameters=7B2026.06 | 18.9 | |
| RKLBackbone=LLaMA, Parameters=7B2026.06 | 18.6 | |
| FKLBackbone=LLaMA, Parameters=7B2026.06 | 18.3 | |
| SeqKDBackbone=LLaMA, Parameters=7B2026.06 | 18.1 | |
| SFT w/o KDBackbone=LLaMA, Parameters=7B2026.06 | 17.5 | |
| TeacherBackbone=GPT-2, Parameters=1.5B2026.06 | 16.6 | |
| ARKDBackbone=GPT-2, Parameters=340M2026.06 | 16.1 | |
| FKL+RKLBackbone=GPT-2, Parameters=340M2026.06 | 15.9 | |
| RKLBackbone=GPT-2, Parameters=340M2026.06 | 15.4 | |
| FKLBackbone=GPT-2, Parameters=340M2026.06 | 15.2 | |
| SeqKDBackbone=GPT-2, Parameters=340M2026.06 | 14.8 | |
| ARKDBackbone=GPT-2, Parameters=120M2026.06 | 14.5 | |
| SFT w/o KDBackbone=GPT-2, Parameters=340M2026.06 | 14.4 | |
| FKL+RKLBackbone=GPT-2, Parameters=120M2026.06 | 13.9 | |
| RKLBackbone=GPT-2, Parameters=120M2026.06 | 13.3 | |
| FKLBackbone=GPT-2, Parameters=120M2026.06 | 12.9 | |
| SFT w/o KDBackbone=GPT-2, Parameters=120M2026.06 | 12.5 | |
| SeqKDBackbone=GPT-2, Parameters=120M2026.06 | 12.2 |