Question Answering on NQ, TriviaQA, PopQA, HotpotQA, 2wiki, MuSiQue, Bamboogle (test)
39.3NQ ScoreTeacher-7B
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Teacher-7BModel Family=Qwen 2.5, Role=Teacher, Size=7B2025.08 | 39.3 | 61 | 39.7 | 37 | 41.4 | 14.6 | 36.8 | 38.5 | |
| DGPOModel Family=Qwen 2.5, Capacity Gap=7B → 0.5B2025.08 | 37.1 | 47.4 | 39.6 | 33.4 | 25.7 | 11.3 | 31.5 | 32.3 | |
| KDModel Family=Qwen 2.5, Capacity Gap=7B → 0.5B2025.08 | 33.8 | 42.8 | 37.1 | 28.8 | 22.3 | 10 | 21 | 28 | |
| PPOModel Family=Qwen 2.5, Capacity Gap=7B → 0.5B2025.08 | 30.6 | 44.4 | 37.9 | 20.5 | 21.8 | 4.1 | 7.3 | 23.8 | |
| Student-0.5BModel Family=Qwen 2.5, Role=Student, Size=0.5B2025.08 | 0.4 | 0.6 | 0.7 | 0.7 | 1.5 | 0 | 0 | 0.6 |