List your company
ResearchFormerly WizWand

Instruction Following on Vicuna (Rouge-L)

20.93Rouge-LTeacher (Qwen1.5)
9.053212.136615.2218.3034Oct 13, 2025Nov 18, 2025Dec 24, 2025Jan 30, 2026Mar 7, 2026Apr 12, 2026May 19, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Teacher (Qwen1.5)Distillation Setting=Cross-Tokenizer KD2026.0320.93
ARMADATeacher=text-to-audio, Distillation Type=L_cosine2026.0319.96
ARMADATeacher=text-to-audio, Distillation Type=L_euclid2026.0319.51
ARMADATeacher=text-to-video, Distillation Type=L_euclid2026.0318.94
ARMADATeacher=text-to-image, Distillation Type=L_cosine2026.0318.89
ARMADATeacher=text-to-video, Distillation Type=L_cosine2026.0318.82
ARMADATeacher=text-to-image, Distillation Type=L_euclid2026.0318.72
ARMADATeacher=text-to-video, Distillation Type=L_elementwise2026.0318.36
ARMADATeacher=text-to-audio, Distillation Type=L_elementwise2026.0318.18
ARMADATeacher=text-to-image, Distillation Type=L_elementwise2026.0318.16
MiniLLMDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.1018.14
SFTModel=LLaMA-3, Role=Teacher, #Param=1.23B2026.0418.01
LLaMA-3.1-8BDistillation Type=undistilled2026.0317.85
DISTILLLENS + MiniLLMPolicy=Hybrid2026.0217.8
AMiDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.1017.69
AMiDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 Medium (0.3B), Student Parameters=0.3B2025.1017.69
CISTStudent=OPT 1.3B, Teacher=OPT 6.7B2026.0517.69
OPT 6.7BRole=Teacher2026.0517.64
MiniLLMDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.1017.62
ABStudent=OPT 1.3B, Teacher=OPT 6.7B2026.0517.59
TAIDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.1017.58
RKLStudent=OPT 1.3B, Teacher=OPT 6.7B2026.0517.54
ABKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.1017.45
ABKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.1017.43
ABKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.1017.36
SRKLStudent=OPT 1.3B, Teacher=OPT 6.7B2026.0517.34
AKLStudent=OPT 1.3B, Teacher=OPT 6.7B2026.0517.26
JSStudent=OPT 1.3B, Teacher=OPT 6.7B2026.0517.15
DistiLLM (SRKL)Distillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.1017.14
DistiLLMPolicy=On2026.0217.1
TAIDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.1017.09
GKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.1017.02
TAIDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.1017.02
Sym-KLStudent=OPT 1.3B, Teacher=OPT 6.7B2026.0516.95
MiniLLMDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.1016.94
DistiLLM (SKL)Distillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.1016.91
MiniLLMPolicy=On2026.0216.9
DistiLLM (SKL)Distillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.1016.85
AMiDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.1016.76
AMiDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 (0.1B), Student Parameters=0.1B2025.1016.76
FKLStudent=OPT 1.3B, Teacher=OPT 6.7B2026.0516.71
SFTDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.1016.64
AMiDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.1016.62
AMiDTeacher Model=GPT-2 XL (1.5B), Student Model=GPT-2 Large (0.8B), Student Parameters=0.8B2025.1016.62
SFTDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.1016.51
KDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.1016.43
SeqKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.1016.42
DistiLLM (SRKL)Distillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.1016.39
SeqKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.1016.35
DistiLLM (SRKL)Distillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.1016.34
GPT-2 1.5BRole=Teacher2026.0516.31
Teacher (GPT-2 XL)Distillation Setting=Same-Tokenizer KD2026.0316.24
DSKD-CMADistillation Setting=Cross-Tokenizer KD, Divergence Function=SRKL2026.0316.21
TeacherModel=GPT-2 XL (1.5B)2025.1016.12
GPT-2 XL (Teacher)Model Role=Teacher, Parameters=1.5B2025.1016.12
DistiLLM (SKL)Distillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.1016.1
CISTStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.0516.1
DSKD-CMA-CTDistillation Setting=Cross-Tokenizer KD, Divergence Function=KL2026.0316.08
ImitKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.1016
AKLDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.1015.98
DSKD-CMA-GADistillation Setting=Cross-Tokenizer KD, Divergence Function=SKL2026.0315.95
AKLDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Large (0.8B), Student Model Parameters=0.8B2025.1015.85
DSKD-CMA-GADistillation Setting=Cross-Tokenizer KD, Divergence Function=KL2026.0315.84
SeqKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.1015.82
RKLStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.0515.82
ABStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.0515.82
DISTILLLENSPolicy=Off2026.0215.8
Sym-KLStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.0515.72
GKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.1015.71
GKDPolicy=On2026.0215.7
MaKDModel=LLaMA-3, Role=Student, #Param=0.69B2026.0415.62
KDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.1015.59
MinEDDistillation Setting=Cross-Tokenizer KD2026.0315.59
ImitKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 Medium (0.3B), Student Model Parameters=0.3B2025.1015.56
DSKDDistillation Setting=Same-Tokenizer KD2026.0315.54
JSStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.0515.47
ImitKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.1015.32
DSKD-CLPDistillation Setting=Cross-Tokenizer KD2026.0315.28
DSKD-CMADistillation Setting=Cross-Tokenizer KD, Divergence Function=KL2026.0315.2
SRKLStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.0514.96
AKLDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.1014.94
KDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.1014.93
DSKD-CLADistillation Setting=Cross-Tokenizer KD2026.0314.93
ULDDistillation Setting=Cross-Tokenizer KD2026.0314.85
SFTDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.1014.79
KD (FKL)Policy=Off2026.0214.7
SFTDistillation Setting=Cross-Tokenizer KD2026.0314.66
GKDDistillation Setup=GPT-2 XL (1.5B) → GPT-2 (0.1B), Student Model Parameters=0.1B2025.1014.6
FKLStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.0514.53
SFTModel=GPT-2, Role=Teacher, #Param=124M2026.0413.99
MaKDModel=GPT-2, Role=Student, #Param=77.4M2026.0413.77
AKLStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.0513.7
KDModel=LLaMA-3, Role=Student, #Param=0.69B2026.0412.62
SeqKDModel=LLaMA-3, Role=Student, #Param=0.69B2026.0412.54
SFTModel=GPT-2, Role=Student, #Param=77.4M2026.0412.24
SeqKDModel=GPT-2, Role=Student, #Param=77.4M2026.0412.14
KDModel=GPT-2, Role=Student, #Param=77.4M2026.0411.82
SFTModel=LLaMA-3, Role=Student, #Param=0.69B2026.0411.38
Student (GPT-2)Distillation Setting=Baseline2026.0310.18
NoneModel=GPT-2, Role=Student, #Param=77.4M2026.049.51