Multiple Choice Question Answering on Average (OBQA, ARC, Riddle, PQA)
68.31Average AccuracyLlama-3.1-8B-Instruct
Evaluation results
| Method | Links | |
|---|---|---|
| Llama-3.1-8B-InstructSetting=Teacher2026.02 | 68.31 | |
| Teacher SelectionSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 67.55 | |
| Similarity-based RouterSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 67.2 | |
| PLM ClassifierSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 66.4 | |
| FLAN-T5 xlargeSetting=Teacher2026.02 | 65.67 | |
| Plackett-Luce RankingSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 64.5 | |
| Knowledge AggregationSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 63.32 | |
| Fine-tuningSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 63.18 | |
| Distilling-Step-by-StepSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 62.76 | |
| TinyLLMSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 62.53 | |
| Teacher SelectionSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 56.68 | |
| Similarity-based RouterSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 56.56 | |
| PLM ClassifierSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 56.04 | |
| Plackett-Luce RankingSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 55.51 | |
| Distilling-Step-by-StepSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 54.23 | |
| Knowledge AggregationSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 53.42 | |
| Fine-tuningSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 53.12 | |
| TinyLLMSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 52.76 | |
| BioMistral-7BSetting=Teacher2026.02 | 49.95 | |
| Llama 2-chatSetting=Teacher2026.02 | 49.05 | |
| InferenceSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 46.69 | |
| Similarity-based RouterSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 45.66 | |
| Teacher SelectionSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 44.63 | |
| PLM ClassifierSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 44.45 | |
| Fine-tuningSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 43.52 | |
| Plackett-Luce RankingSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 42.49 | |
| TinyLLMSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 42.38 | |
| Knowledge AggregationSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 42.01 | |
| Distilling-Step-by-StepSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 41.47 | |
| InferenceSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 32.7 | |
| InferenceSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 19.31 |