List your company
ResearchFormerly WizWand

Instruction Following on Dolly

71.3ScoreOurs 12
17.552831.506445.4659.4136Oct 5, 2023Mar 15, 2024Aug 25, 2024Feb 3, 2025Jul 16, 2025Dec 25, 2025Jun 6, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Ours 12Distillation Method=Random Sampling KD, Number of Unique Tokens=12, Teacher Model Size=8B, Student Model Size=3B, Evaluation Judge=Llama 3.1 405B Instruct2025.0371.3
FullKDDistillation Method=Full KD, Teacher Model Size=8B, Student Model Size=3B, Evaluation Judge=Llama 3.1 405B Instruct2025.0366.1
Top-K 50Distillation Method=Top-K KD, Number of Unique Tokens=50, Teacher Model Size=8B, Student Model Size=3B, Evaluation Judge=Llama 3.1 405B Instruct2025.0365.4
CEDistillation Method=None (Cross-Entropy), Teacher Model Size=8B, Student Model Size=3B, Evaluation Judge=Llama 3.1 405B Instruct2025.0364.2
Top-K 12Distillation Method=Top-K KD, Number of Unique Tokens=12, Teacher Model Size=8B, Student Model Size=3B, Evaluation Judge=Llama 3.1 405B Instruct2025.0359
FedAvg+AlignFedBackbone=Llama3-8B2026.0658.52
FedAvgBackbone=Llama3-8B2026.0658.49
FFA-LoRA+AlignFedBackbone=Llama3-8B2026.0658.32
FFA-LoRABackbone=Llama3-8B2026.0658.21
FedSA-LoRA+AlignFedBackbone=Llama3-8B2026.0658.13
FedSA-LoRABackbone=Llama3-8B2026.0658
FFA-LoRA+FedBuffBackbone=Llama3-8B2026.0657.94
FedAvg+FedBuffBackbone=Llama3-8B2026.0657.89
FedSA-LoRA+FedBuffBackbone=Llama3-8B2026.0657.71
LumiNetModel Architecture=GPT-2, Parameters=760M2023.1028.6
LumiNetModel Architecture=GPT-2, Parameters=340M2023.1027.8
TeacherModel Architecture=Teacher, Parameters=1.5B2023.1027.6
KDModel Architecture=GPT-2, Parameters=760M2023.1025.9
SeqKDModel Architecture=GPT-2, Parameters=760M2023.1025.6
SFT w/o KDModel Architecture=GPT-2, Parameters=340M2023.1025.5
SFT w/o KDModel Architecture=GPT-2, Parameters=760M2023.1025.4
SeqKDModel Architecture=GPT-2, Parameters=340M2023.1025.3
KDModel Architecture=GPT-2, Parameters=340M2023.1025
LumiNetModel Architecture=GPT-2, Parameters=120M2023.1023.8
SFT w/o KDModel Architecture=GPT-2, Parameters=120M2023.1023.3
CBDParams=134.002026.0522.99
KDModel Architecture=GPT-2, Parameters=120M2023.1022.8
SeqKDModel Architecture=GPT-2, Parameters=120M2023.1022.7
CBDParams=277.002026.0520.82
Van-LearngeneParams=297.242026.0520.67
Auto-LearngeneParams=297.242026.0520.43
Auto-LearngeneParams=174.282026.0520.3
Auto-LearngeneParams=235.762026.0520.03
Van-LearngeneParams=235.762026.0519.92
CBDParams=220.002026.0519.82
Van-LearngeneParams=174.282026.0519.62