Instruction Following on UnNI (OOD)
39Rouge-LTeacher
Evaluation Results
| Method | Links | |
|---|---|---|
| TeacherBackbone=LLaMA, Parameters=13B2026.06 | 39 | |
| ARKDBackbone=LLaMA, Parameters=7B2026.06 | 38.1 | |
| FKL+RKLBackbone=LLaMA, Parameters=7B2026.06 | 37.5 | |
| RKLBackbone=LLaMA, Parameters=7B2026.06 | 37.1 | |
| SeqKDBackbone=LLaMA, Parameters=7B2026.06 | 36.6 | |
| FKLBackbone=LLaMA, Parameters=7B2026.06 | 36.5 | |
| SFT w/o KDBackbone=LLaMA, Parameters=7B2026.06 | 35.8 | |
| TeacherBackbone=GPT-2, Parameters=1.5B2026.06 | 31.6 | |
| ARKDBackbone=GPT-2, Parameters=340M2026.06 | 28.4 | |
| FKL+RKLBackbone=GPT-2, Parameters=340M2026.06 | 27.8 | |
| RKLBackbone=GPT-2, Parameters=340M2026.06 | 27.1 | |
| FKLBackbone=GPT-2, Parameters=340M2026.06 | 26.6 | |
| SeqKDBackbone=GPT-2, Parameters=340M2026.06 | 26 | |
| SFT w/o KDBackbone=GPT-2, Parameters=340M2026.06 | 25.7 | |
| ARKDBackbone=GPT-2, Parameters=120M2026.06 | 21.9 | |
| FKL+RKLBackbone=GPT-2, Parameters=120M2026.06 | 20.3 | |
| RKLBackbone=GPT-2, Parameters=120M2026.06 | 19.8 | |
| FKLBackbone=GPT-2, Parameters=120M2026.06 | 19.2 | |
| SeqKDBackbone=GPT-2, Parameters=120M2026.06 | 18.5 | |
| SFT w/o KDBackbone=GPT-2, Parameters=120M2026.06 | 18.4 |