Reasoned Question Answering on MMLU Pro (Subject Specific Scores)
33.17MMLU Pro Score: Computer ScienceD&R
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| D&RDistillation Strategy=Multi-Teacher Distillation, Training Protocol=Full (with T-DPO)2025.06 | 33.17 | 34.77 | 67.36 | |
| D&RSFTDistillation Strategy=Multi-Teacher Distillation, Training Protocol=SFT-only2025.06 | 32.68 | 29.23 | 64.44 | |
| Single TeacherClaude 3.5Distillation Strategy=Single-Teacher Distillation, Teacher Model=Claude 3.52025.06 | 30.73 | 30.77 | 62.76 | |
| Single TeacherGPT-4oDistillation Strategy=Single-Teacher Distillation, Teacher Model=GPT-4o2025.06 | 27.8 | 28.63 | 62.34 | |
| Single TeacherGemini 1.5 ProDistillation Strategy=Single-Teacher Distillation, Teacher Model=Gemini 1.5 Pro2025.06 | 25.85 | 24.92 | 60.67 | |
| MAGDIDistillation Strategy=Multi-Teacher Distillation2025.06 | 22.93 | 20 | 54.39 | |
| Mistral-7B-InstructDistillation Strategy=No Distillation2025.06 | 20.49 | 18.46 | 48.95 |