List your company
ResearchFormerly WizWand

Language Understanding on MMLU (Accuracy and Acc Increase)

62.38AccuracyTuneShift-KD
40.550446.217751.88557.5523Mar 25, 2026Apr 2, 2026Apr 10, 2026Apr 18, 2026Apr 26, 2026May 4, 2026May 12, 2026
Updated 5mo ago

Evaluation results

MethodLinks
TuneShift-KDSource=Gemma-2B, Target=Gemma-7B, Evaluation Setting=Target LoRA2026.0362.380.76—
TuneShift-KDSource=Gemma-2B, Target=Gemma-7B, Evaluation Setting=Target (no LoRA)2026.0361.62——
Trans-LoRASource=Gemma-2B, Target=Gemma-7B, Evaluation Setting=Target LoRA2026.0361.230.78—
Trans-LoRASource=Gemma-2B, Target=Gemma-7B, Evaluation Setting=Target (no LoRA)2026.0360.45——
Trans-LoRASource=Llama2-7B, Target=Llama2-13B, Evaluation Setting=Target LoRA2026.0355.091.37—
Trans-LoRASource=Llama2-7B, Target=Llama2-13B, Evaluation Setting=Target (no LoRA)2026.0353.72——
TuneShift-KDSource=Llama2-7B, Target=Llama2-13B, Evaluation Setting=Target LoRA2026.0353.620.49—
TuneShift-KDSource=Llama2-7B, Target=Llama2-13B, Evaluation Setting=Target (no LoRA)2026.0353.13——
RL+SAEModel=Qwen3-30B-A3B2026.0552.23——
Before RLModel=Qwen3-30B-A3B2026.0551.75——
Vanilla RLModel=Qwen3-30B-A3B2026.0551.43——
Vanilla RLModel=Qwen3-8B2026.0548.55——
Before RLModel=Qwen3-8B2026.0548.4——
RL+SAEModel=Qwen3-8B2026.0548.4——
TuneShift-KDSource=Llama2-7B, Target=Llama2-13B, Evaluation Setting=Source LoRA2026.0347.19——
Trans-LoRASource=Llama2-7B, Target=Llama2-13B, Evaluation Setting=Source LoRA2026.0345.89——
Trans-LoRASource=Gemma-2B, Target=Gemma-7B, Evaluation Setting=Source LoRA2026.0342.34——
Vanilla RLModel=Qwen3-1.7B2026.0541.83——
Before RLModel=Qwen3-1.7B2026.0541.78——
RL+SAEModel=Qwen3-1.7B2026.0541.67——
TuneShift-KDSource=Gemma-2B, Target=Gemma-7B, Evaluation Setting=Source LoRA2026.0341.39——
BF16Backbone=TinyLlama-1.1B, Quantization=BF16, Subjects=57, Limit=3002026.05——24.89
BF16Backbone=Qwen2.5-3B, Quantization=BF16, Subjects=57, Limit=3002026.05——67
BF16Backbone=Mistral-7B, Quantization=BF16, Subjects=57, Limit=3002026.05——61.5
QAM-W-5.5Backbone=TinyLlama-1.1B, Quantization=QAM-W-5.5, Subjects=57, Limit=3002026.05—1825.07
QAM-W-5.5Backbone=Qwen2.5-3B, Quantization=QAM-W-5.5, Subjects=57, Limit=3002026.05—1566.85
QAM-W-5.5Backbone=Mistral-7B, Quantization=QAM-W-5.5, Subjects=57, Limit=3002026.05—361.53