Multiple Choice Question Answering on RiddleSense
70.59AccuracySimilarity-based Router
Evaluation results
| Method | Links | |
|---|---|---|
| Similarity-based RouterSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 70.59 | |
| Teacher SelectionSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 70.39 | |
| Plackett-Luce RankingSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 69.41 | |
| Knowledge AggregationSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 68.63 | |
| Distilling-Step-by-StepSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 68.43 | |
| PLM ClassifierSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 68.24 | |
| Fine-tuningSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 67.45 | |
| TinyLLMSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 67.25 | |
| Teacher SelectionSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 61.76 | |
| Llama-3.1-8B-InstructSetting=Teacher2026.02 | 60.98 | |
| Similarity-based RouterSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 60.78 | |
| Distilling-Step-by-StepSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 59.41 | |
| PLM ClassifierSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 59.22 | |
| TinyLLMSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 58.82 | |
| Plackett-Luce RankingSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 58.63 | |
| Knowledge AggregationSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 58.43 | |
| Fine-tuningSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 56.86 | |
| FLAN-T5 xlargeSetting=Teacher2026.02 | 53.73 | |
| Similarity-based RouterSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 49.61 | |
| Fine-tuningSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 49.22 | |
| PLM ClassifierSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 49.22 | |
| Teacher SelectionSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 48.82 | |
| Plackett-Luce RankingSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 45.69 | |
| Distilling-Step-by-StepSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 44.9 | |
| TinyLLMSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 44.12 | |
| Knowledge AggregationSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 43.92 | |
| Llama 2-chatSetting=Teacher2026.02 | 43.73 | |
| InferenceSetting=Student, Backbone=FLAN-T5 large (783M)2026.02 | 39.8 | |
| InferenceSetting=Student, Backbone=FLAN-T5 base (248M)2026.02 | 30.78 | |
| BioMistral-7BSetting=Teacher2026.02 | 23.14 | |
| InferenceSetting=Student, Backbone=FLAN-T5 small (77M)2026.02 | 13.33 |