Language Modeling on LAMBADA OpenAI (Accuracy, AR%)
59.3AccuracyTeacher (Qwen3-4B)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Teacher (Qwen3-4B)Precision=BF162026.06 | 59.3 | — | |
| Muon + PCZero-shot=true, Model=Llama-1B2026.06 | 58.12 | — | |
| Muon baselineZero-shot=true, Model=Llama-1B2026.06 | 57.17 | — | |
| Recover-LoRA (OpenHermes-10k)Recovery Training Dataset=OpenHermes-10k, Quantization=W2-GateUp2026.06 | 55.6 | 93.3 | |
| AdamW + PCZero-shot=true, Model=Llama-1B2026.06 | 53.66 | — | |
| Recover-LoRA (Synthetic-10k)Recovery Training Dataset=Synthetic-10k, Quantization=W2-GateUp2026.06 | 53.4 | 89.5 | |
| AdamW baselineZero-shot=true, Model=Llama-1B2026.06 | 50.26 | — | |
| Quantized Student (Qwen3-4B)Quantization=W2-GateUp (gate/up at INT2, remaining at BF16)2026.06 | 3.4 | — |