General Language Understanding on Winogrande, HellaSwag, ARC, MMLU Consolidated
71.09Average AccuracyTeacher (DeepSeek-V2-Lite)
Evaluation results
| Method | Links | |
|---|---|---|
| Teacher (DeepSeek-V2-Lite)2026.05 | 71.09 | |
| DO-ACPScoring=DO-ACP, K=62026.05 | 42.39 | |
| SFScoring=SF, K=122026.05 | 41.16 | |
| DO-ACPScoring=DO-ACP, K=122026.05 | 41.07 | |
| ACPScoring=ACP, K=122026.05 | 40.93 | |
| CPScoring=CP, K=122026.05 | 40.53 | |
| ACPScoring=ACP, K=62026.05 | 40.37 | |
| SFScoring=SF, K=62026.05 | 40.04 | |
| CPScoring=CP, K=62026.05 | 38.07 | |
| Random initialization2026.05 | 30.27 | |
| Random FFN + teacher attn2026.05 | 30.25 |