Mathematical Reasoning on GSM8K (Avg@8, Pass@8, Avg Response Length)
83.5Avg@8Full-vocab KL OPD
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Full-vocab KL OPDTeacher=Qwen3-4B, Student=Qwen3-1.7B-Base2026.06 | 83.5 | 95 | 3,954 | |
| PowerOPDTeacher=Qwen3-4B, Student=Qwen3-1.7B-Base, alpha=5002026.06 | 83.13 | 94.31 | 316 | |
| PowerOPDTeacher=Qwen3-4B, Student=Qwen3-1.7B-Base, alpha=502026.06 | 83.11 | 94.24 | 313 | |
| PowerOPDTeacher=Qwen3-4B, Student=Qwen3-1.7B-Base, alpha=102026.06 | 83.01 | 94.69 | 315 | |
| PowerOPDTeacher=Qwen3-4B, Student=Qwen3-1.7B-Base, alpha=1002026.06 | 82.92 | 93.33 | 307 | |
| PowerOPDTeacher=Qwen3-4B, Student=Qwen3-1.7B-Base, alpha=52026.06 | 82.68 | 95.07 | 315 | |
| PowerOPDTeacher=Qwen3-4B, Student=Qwen3-1.7B-Base, alpha=0.12026.06 | 82.46 | 94.77 | 316 | |
| Vanilla OPD + TanhTeacher=Qwen3-4B, Student=Qwen3-1.7B-Base2026.06 | 82.08 | 94.62 | 331 | |
| PowerOPDTeacher=Qwen3-4B, Student=Qwen3-1.7B-Base, alpha=12026.06 | 82.04 | 95.38 | 317 | |
| Vanilla OPD + ClipTeacher=Qwen3-4B, Student=Qwen3-1.7B-Base2026.06 | 80.36 | 94.01 | 326 | |
| Vanilla OPDTeacher=Qwen3-4B, Student=Qwen3-1.7B-Base2026.06 | 73.93 | 95.68 | 351 | |
| Vanilla OPD + Z-ScoreTeacher=Qwen3-4B, Student=Qwen3-1.7B-Base2026.06 | 42.92 | 89.61 | 372 |