Knowledge Evaluation on MMLU STEM
49.7AccuracyTSD-KD
Evaluation results
| Method | Links | |
|---|---|---|
| TSD-KDFine-tuning=Parameter Efficient Finetuning (PEFT), Backbone=Qwen2.5, PEFT_Method=LoRA2026.02 | 49.7 | |
| MiniLLMFine-tuning=Parameter Efficient Finetuning (PEFT), Backbone=Qwen2.5, PEFT_Method=LoRA2026.02 | 47.9 | |
| GKDFine-tuning=Parameter Efficient Finetuning (PEFT), Backbone=Qwen2.5, PEFT_Method=LoRA, beta=0.92026.02 | 47.6 | |
| DistiLLMFine-tuning=Parameter Efficient Finetuning (PEFT), Backbone=Qwen2.5, PEFT_Method=LoRA2026.02 | 47.3 | |
| Supervised-KDFine-tuning=Parameter Efficient Finetuning (PEFT), Backbone=Qwen2.5, PEFT_Method=LoRA2026.02 | 47.2 |