List your company
ResearchFormerly WizWand

Instruction-following on (val)

34.39ROUGE-LAMiD (Ours)
26.766828.745930.72532.7041Oct 13, 2025
Updated 8mo ago

Evaluation results

MethodLinks
AMiD (Ours)Distillation Pair=OpenLLaMA2-7B → OpenLLaMA2-3B2025.1034.39
ABKDDistillation Pair=OpenLLaMA2-7B → OpenLLaMA2-3B2025.1033.91
DistiLLM (SRKL)Distillation Pair=OpenLLaMA2-7B → OpenLLaMA2-3B2025.1033.18
DistiLLM (SKL)Distillation Pair=OpenLLaMA2-7B → OpenLLaMA2-3B2025.1033.07
AMiDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 Large (0.8B), Student Parameters=0.8B2025.1031.1
TAIDDistillation Pair=OpenLLaMA2-7B → OpenLLaMA2-3B2025.1030.85
AMiDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 Medium (0.3B), Student Parameters=0.3B2025.1030.83
DistiLLM (SRKL)Teacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 Large (0.8B), Student Parameters=0.8B2025.1030.59
ABKDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 Large (0.8B), Student Parameters=0.8B2025.1030.49
TAIDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 Large (0.8B), Student Parameters=0.8B2025.1029.83
DistiLLM (SRKL)Teacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 Medium (0.3B), Student Parameters=0.3B2025.1029.72
DistiLLM (SKL)Teacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 Large (0.8B), Student Parameters=0.8B2025.1029.69
DistiLLM (SKL)Teacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 Medium (0.3B), Student Parameters=0.3B2025.1029.65
ABKDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 Medium (0.3B), Student Parameters=0.3B2025.1029.64
TAIDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 Medium (0.3B), Student Parameters=0.3B2025.1029.45
GKDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 Large (0.8B), Student Parameters=0.8B2025.1029.36
AMiDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 (0.1B), Student Parameters=0.1B2025.1029.24
ABKDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 (0.1B), Student Parameters=0.1B2025.1028.61
TAIDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 (0.1B), Student Parameters=0.1B2025.1028.37
DistiLLM (SRKL)Teacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 (0.1B), Student Parameters=0.1B2025.1028.21
GKDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 Medium (0.3B), Student Parameters=0.3B2025.1027.9
DistiLLM (SKL)Teacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 (0.1B), Student Parameters=0.1B2025.1027.88
GKDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 (0.1B), Student Parameters=0.1B2025.1027.06