List your company
ResearchFormerly WizWand

Instruction Following on Dolly (Rouge-L)

27.74Rouge-LCIST
-0.58966.765214.1221.4748Feb 14, 2026Mar 1, 2026Mar 17, 2026Apr 2, 2026Apr 17, 2026May 3, 2026May 19, 2026
Updated 5mo ago

Evaluation results

MethodLinks
CISTStudent=OPT 1.3B, Teacher=OPT 6.7B2026.0527.74
OPT 6.7BRole=Teacher2026.0527.52
SFTModel=LLaMA-3, Role=Teacher, #Param=1.23B2026.0427.47
Teacher (Qwen1.5)Distillation Setting=Cross-Tokenizer KD2026.0327.45
Teacher (GPT-2 XL)Distillation Setting=Same-Tokenizer KD2026.0327.07
GPT-2 1.5BRole=Teacher2026.0527
ABStudent=OPT 1.3B, Teacher=OPT 6.7B2026.0526.86
RKLStudent=OPT 1.3B, Teacher=OPT 6.7B2026.0526.58
JSStudent=OPT 1.3B, Teacher=OPT 6.7B2026.0526.39
AKLStudent=OPT 1.3B, Teacher=OPT 6.7B2026.0526.2
FKLStudent=OPT 1.3B, Teacher=OPT 6.7B2026.0526.07
SRKLStudent=OPT 1.3B, Teacher=OPT 6.7B2026.0526.04
DSKDDistillation Setting=Same-Tokenizer KD2026.0325.98
Sym-KLStudent=OPT 1.3B, Teacher=OPT 6.7B2026.0525.73
CISTStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.0525.42
DISTILLLENSPolicy=Off2026.0225.2
DISTILLLENS + MiniLLMPolicy=Hybrid2026.0225.1
DSKD-CMA-GADistillation Setting=Cross-Tokenizer KD, Divergence Function=KL2026.0324.92
DSKD-CMA-GADistillation Setting=Cross-Tokenizer KD, Divergence Function=SKL2026.0324.79
RKLStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.0524.67
DSKD-CMADistillation Setting=Cross-Tokenizer KD, Divergence Function=SRKL2026.0324.64
DSKD-CMA-CTDistillation Setting=Cross-Tokenizer KD, Divergence Function=KL2026.0324.63
MiniLLMPolicy=On2026.0224.6
DSKD-CLPDistillation Setting=Cross-Tokenizer KD2026.0324.52
SRKLStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.0524.48
GKDPolicy=On2026.0224.4
Sym-KLStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.0524.38
ABStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.0524.32
DSKD-CLADistillation Setting=Cross-Tokenizer KD2026.0324.2
DSKD-CMADistillation Setting=Cross-Tokenizer KD, Divergence Function=KL2026.0324.13
DistiLLMPolicy=On2026.0224.1
JSStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.0524.08
FKLStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.0523.8
KD (FKL)Policy=Off2026.0223.5
SFTDistillation Setting=Cross-Tokenizer KD2026.0322.83
SFTModel=GPT-2, Role=Teacher, #Param=124M2026.0422.5
AKLStudent=GPT-2 0.1B, Teacher=GPT-2 1.5B2026.0521.83
ULDDistillation Setting=Cross-Tokenizer KD2026.0321.73
MinEDDistillation Setting=Cross-Tokenizer KD2026.0321.69
MaKDModel=LLaMA-3, Role=Student, #Param=0.69B2026.0420.74
MaKDModel=GPT-2, Role=Student, #Param=77.4M2026.0418.56
SeqKDModel=LLaMA-3, Role=Student, #Param=0.69B2026.0417.05
SeqKDModel=GPT-2, Role=Student, #Param=77.4M2026.0416.56
KDModel=GPT-2, Role=Student, #Param=77.4M2026.0416.25
SFTModel=LLaMA-3, Role=Student, #Param=0.69B2026.0416.1
KDModel=LLaMA-3, Role=Student, #Param=0.69B2026.0416.04
SFTModel=GPT-2, Role=Student, #Param=77.4M2026.0415.47
Student (GPT-2)Distillation Setting=Baseline2026.036.39
NoneModel=GPT-2, Role=Student, #Param=77.4M2026.045.69
NoneModel=LLaMA-3, Role=Student, #Param=0.69B2026.040.5