General Knowledge QA on MMLU-Pro
69.4AccuracyDistillable
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DistillableBase Model=Qwen3-8B2026.04 | 69.4 | 70.7 | 82.8 | |
| UndistillableBase Model=Qwen3-8B2026.04 | 68.3 | 68.7 | 82.5 | |
| TeacherBase Model=Qwen3-8B2026.04 | 65.5 | 68.1 | 81.5 | |
| DistillableBase Model=Gemma-3-12B2026.04 | 63.1 | 62.7 | 85.7 | |
| UndistillableBase Model=Gemma-3-12B2026.04 | 60.8 | 59.3 | 86.8 | |
| ImproveBase Model=Qwen3-1.7B2026.04 | 54.6 | 54.4 | 75.6 | |
| TeacherBase Model=Gemma-3-12B2026.04 | 52.8 | 51.1 | 80.1 | |
| GKD-FKLBase Model=Qwen3-1.7B2026.04 | 45.8 | 43.7 | 75.6 | |
| GKD-RKLBase Model=Qwen3-1.7B2026.04 | 45.7 | 47.6 | 68.1 | |
| SFTBase Model=Qwen3-1.7B2026.04 | 44.6 | 43.4 | 77.8 | |
| ImproveBase Model=Gemma-3-4B2026.04 | 40 | 36.5 | 73.5 | |
| GKD-RKLBase Model=Gemma-3-4B2026.04 | 39.2 | 36 | 73.2 | |
| GKD-FKLBase Model=Gemma-3-4B2026.04 | 38.9 | 34.8 | 72.5 | |
| SFTBase Model=Gemma-3-4B2026.04 | 36.3 | 32.9 | 68.4 | |
| StudentBase Model=Gemma-3-4B2026.04 | 34.4 | 30.8 | 68 | |
| StudentBase Model=Qwen3-1.7B2026.04 | 29.7 | 26.6 | 68.1 | |
| MisledBase Model=Gemma-3-4B2026.04 | 27.5 | 19.2 | 44.8 | |
| MisledBase Model=Qwen3-1.7B2026.04 | 12.3 | 8.4 | 38.3 |