Instruction Following on Super NI
32.64ROUGE-LAMiD
Evaluation Results
| Method | Links | |
|---|---|---|
| AMiDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 Large (0.8B), Student Parameters=0.8B2025.10 | 32.64 | |
| OPT 6.7BRole=Teacher2026.05 | 30.41 | |
| AMiDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 (0.1B), Student Parameters=0.1B2025.10 | 29.71 | |
| AMiDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 Medium (0.3B), Student Parameters=0.3B2025.10 | 29.04 | |
| GPT-2 XL (Teacher)Model Role=Teacher, Parameters=1.5B2025.10 | 27.21 | |
| GPT-2 1.5BRole=Teacher2026.05 | 26.46 | |
| CISTStudent=OPT 1.3B, Teacher=OPT 6.7B2026.05 | 25.19 | |
| ABStudent=OPT 1.3B, Teacher=OPT 6.7B2026.05 | 24.39 | |
| JSStudent=OPT 1.3B, Teacher=OPT 6.7B2026.05 | 24.02 | |
| Sym-KLStudent=OPT 1.3B, Teacher=OPT 6.7B2026.05 | 23.4 | |
| SRKLStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.05 | 23.25 | |
| ABStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.05 | 23.08 | |
| SRKLStudent=OPT 1.3B, Teacher=OPT 6.7B2026.05 | 22.96 | |
| RKLStudent=OPT 1.3B, Teacher=OPT 6.7B2026.05 | 22.64 | |
| AKLStudent=OPT 1.3B, Teacher=OPT 6.7B2026.05 | 22.12 | |
| FKLStudent=OPT 1.3B, Teacher=OPT 6.7B2026.05 | 22.11 | |
| CISTStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.05 | 21.81 | |
| RKLStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.05 | 21.03 | |
| Sym-KLStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.05 | 19.92 | |
| JSStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.05 | 19.66 | |
| FKLStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.05 | 16.27 | |
| AKLStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.05 | 15.4 |