General Reasoning and Knowledge on Aggregate (BBH, MMLU, ARC-C, ThmQA)
26.27Mean ScoreHybKD
Evaluation Results
| Method | Links | |
|---|---|---|
| HybKDDistillation Pair=Llama3.1-8B → Llama3.2-1B2026.05 | 26.27 | |
| α-β divergenceDistillation Pair=Llama3.1-8B → Llama3.2-1B2026.05 | 25.01 | |
| JS divergenceDistillation Pair=Llama3.1-8B → Llama3.2-1B2026.05 | 24.84 | |
| Adaptive KLDistillation Pair=Llama3.1-8B → Llama3.2-1B2026.05 | 24.41 | |
| Skew RKLDistillation Pair=Llama3.1-8B → Llama3.2-1B2026.05 | 23.69 | |
| Skew FKLDistillation Pair=Llama3.1-8B → Llama3.2-1B2026.05 | 23.37 | |
| Reverse KLDistillation Pair=Llama3.1-8B → Llama3.2-1B2026.05 | 22.75 | |
| Total VariationDistillation Pair=Llama3.1-8B → Llama3.2-1B2026.05 | 20.66 | |
| HybKDDistillation Pair=Gemma3-4B → Gemma3-1B2026.05 | 16.26 | |
| Skew FKLDistillation Pair=Gemma3-4B → Gemma3-1B2026.05 | 15.81 | |
| α-β divergenceDistillation Pair=Gemma3-4B → Gemma3-1B2026.05 | 15.79 | |
| Reverse KLDistillation Pair=Gemma3-4B → Gemma3-1B2026.05 | 15.78 | |
| JS divergenceDistillation Pair=Gemma3-4B → Gemma3-1B2026.05 | 15.75 | |
| Skew RKLDistillation Pair=Gemma3-4B → Gemma3-1B2026.05 | 15.08 | |
| Adaptive KLDistillation Pair=Gemma3-4B → Gemma3-1B2026.05 | 14.63 | |
| Total VariationDistillation Pair=Gemma3-4B → Gemma3-1B2026.05 | 14.62 |