Instruction Following on Instruction-Following (val)
31.1ROUGE-LAMiD
Evaluation Results
| Method | Links | |
|---|---|---|
| AMiDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.10 | 31.1 | |
| AMiDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.10 | 30.83 | |
| DistiLLM (SRKL)Distillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.10 | 30.59 | |
| ABKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.10 | 30.49 | |
| TAIDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.10 | 29.83 | |
| DistiLLM (SRKL)Distillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.10 | 29.72 | |
| DistiLLM (SKL)Distillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.10 | 29.69 | |
| DistiLLM (SKL)Distillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.10 | 29.65 | |
| ABKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.10 | 29.64 | |
| TAIDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.10 | 29.45 | |
| GKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.10 | 29.36 | |
| AMiDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.10 | 29.24 | |
| ABKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.10 | 28.61 | |
| KDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.10 | 28.52 | |
| SFTDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.10 | 28.48 | |
| SeqKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.10 | 28.41 | |
| TAIDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.10 | 28.37 | |
| SeqKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.10 | 28.24 | |
| DistiLLM (SRKL)Distillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.10 | 28.21 | |
| SFTDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.10 | 27.96 | |
| GKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.10 | 27.9 | |
| DistiLLM (SKL)Distillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.10 | 27.88 | |
| AKLDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.10 | 27.81 | |
| AKLDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.10 | 27.69 | |
| GKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.10 | 27.06 | |
| ImitKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.10 | 26.96 | |
| SeqKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.10 | 26.07 | |
| KDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.10 | 26.03 | |
| ImitKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.10 | 25.93 | |
| SFTDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.10 | 25.81 | |
| AKLDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.10 | 25.62 | |
| KDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.10 | 25.25 | |
| ImitKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.10 | 23.91 |