Knowledge on MMLU-R
46.1AccuracyTeacher (Qwen3-0.6B)
Evaluation results
| Method | Links | |
|---|---|---|
| Teacher (Qwen3-0.6B)Role=Teacher, Model=Qwen3-0.6B, Evaluation Protocol=generation-based evaluation2026.03 | 46.1 | |
| Hybrid KDASequence Mixer=KDA, Attention Layers=7, Evaluation Protocol=generation-based evaluation2026.03 | 40.3 | |
| Hybrid MambaSequence Mixer=Mamba2, Attention Layers=7, Evaluation Protocol=generation-based evaluation2026.03 | 38.6 | |
| Pure KDASequence Mixer=KDA, Attention Layers=0, Evaluation Protocol=generation-based evaluation2026.03 | 29.4 | |
| Pure MambaSequence Mixer=Mamba2, Attention Layers=0, Evaluation Protocol=generation-based evaluation2026.03 | 18 |