Language Understanding on MMLU (Accuracy and Acc Increase)
62.38AccuracyTuneShift-KD
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| TuneShift-KDSource=Gemma-2B, Target=Gemma-7B, Evaluation Setting=Target LoRA2026.03 | 62.38 | 0.76 | — | |
| TuneShift-KDSource=Gemma-2B, Target=Gemma-7B, Evaluation Setting=Target (no LoRA)2026.03 | 61.62 | — | — | |
| Trans-LoRASource=Gemma-2B, Target=Gemma-7B, Evaluation Setting=Target LoRA2026.03 | 61.23 | 0.78 | — | |
| Trans-LoRASource=Gemma-2B, Target=Gemma-7B, Evaluation Setting=Target (no LoRA)2026.03 | 60.45 | — | — | |
| Trans-LoRASource=Llama2-7B, Target=Llama2-13B, Evaluation Setting=Target LoRA2026.03 | 55.09 | 1.37 | — | |
| Trans-LoRASource=Llama2-7B, Target=Llama2-13B, Evaluation Setting=Target (no LoRA)2026.03 | 53.72 | — | — | |
| TuneShift-KDSource=Llama2-7B, Target=Llama2-13B, Evaluation Setting=Target LoRA2026.03 | 53.62 | 0.49 | — | |
| TuneShift-KDSource=Llama2-7B, Target=Llama2-13B, Evaluation Setting=Target (no LoRA)2026.03 | 53.13 | — | — | |
| RL+SAEModel=Qwen3-30B-A3B2026.05 | 52.23 | — | — | |
| Before RLModel=Qwen3-30B-A3B2026.05 | 51.75 | — | — | |
| Vanilla RLModel=Qwen3-30B-A3B2026.05 | 51.43 | — | — | |
| Vanilla RLModel=Qwen3-8B2026.05 | 48.55 | — | — | |
| Before RLModel=Qwen3-8B2026.05 | 48.4 | — | — | |
| RL+SAEModel=Qwen3-8B2026.05 | 48.4 | — | — | |
| TuneShift-KDSource=Llama2-7B, Target=Llama2-13B, Evaluation Setting=Source LoRA2026.03 | 47.19 | — | — | |
| Trans-LoRASource=Llama2-7B, Target=Llama2-13B, Evaluation Setting=Source LoRA2026.03 | 45.89 | — | — | |
| Trans-LoRASource=Gemma-2B, Target=Gemma-7B, Evaluation Setting=Source LoRA2026.03 | 42.34 | — | — | |
| Vanilla RLModel=Qwen3-1.7B2026.05 | 41.83 | — | — | |
| Before RLModel=Qwen3-1.7B2026.05 | 41.78 | — | — | |
| RL+SAEModel=Qwen3-1.7B2026.05 | 41.67 | — | — | |
| TuneShift-KDSource=Gemma-2B, Target=Gemma-7B, Evaluation Setting=Source LoRA2026.03 | 41.39 | — | — | |
| BF16Backbone=TinyLlama-1.1B, Quantization=BF16, Subjects=57, Limit=3002026.05 | — | — | 24.89 | |
| BF16Backbone=Qwen2.5-3B, Quantization=BF16, Subjects=57, Limit=3002026.05 | — | — | 67 | |
| BF16Backbone=Mistral-7B, Quantization=BF16, Subjects=57, Limit=3002026.05 | — | — | 61.5 | |
| QAM-W-5.5Backbone=TinyLlama-1.1B, Quantization=QAM-W-5.5, Subjects=57, Limit=3002026.05 | — | 18 | 25.07 | |
| QAM-W-5.5Backbone=Qwen2.5-3B, Quantization=QAM-W-5.5, Subjects=57, Limit=3002026.05 | — | 15 | 66.85 | |
| QAM-W-5.5Backbone=Mistral-7B, Quantization=QAM-W-5.5, Subjects=57, Limit=3002026.05 | — | 3 | 61.53 |