Biomedical reasoning on BioASQ out-of-domain
91.87AccuracySimilarity-based Router
Evaluation results
| Method | Links | |
|---|---|---|
| Similarity-based RouterSetting=FLAN-T5 large (783M) as Student2026.02 | 91.87 | |
| BioMistral-7BSetting=Teacher2026.02 | 90.24 | |
| Similarity-based RouterSetting=FLAN-T5 base (248M) as Student2026.02 | 90.24 | |
| Fine-tuningSetting=FLAN-T5 large (783M) as Student2026.02 | 90.24 | |
| Fine-tuningSetting=FLAN-T5 base (248M) as Student2026.02 | 89.43 | |
| Llama-3.1-8B-InstructSetting=Teacher2026.02 | 88.62 | |
| Plackett-Luce RankingSetting=FLAN-T5 large (783M) as Student2026.02 | 87.8 | |
| Plackett-Luce RankingSetting=FLAN-T5 base (248M) as Student2026.02 | 86.99 | |
| Distilling-Step-by-StepSetting=FLAN-T5 large (783M) as Student2026.02 | 86.99 | |
| Distilling-Step-by-StepSetting=FLAN-T5 base (248M) as Student2026.02 | 86.18 | |
| PLM ClassifierSetting=FLAN-T5 base (248M) as Student2026.02 | 85.37 | |
| PLM ClassifierSetting=FLAN-T5 large (783M) as Student2026.02 | 83.74 | |
| TinyLLMSetting=FLAN-T5 large (783M) as Student2026.02 | 82.93 | |
| PLM ClassifierSetting=FLAN-T5 small (77M) as Student2026.02 | 82.11 | |
| Similarity-based RouterSetting=FLAN-T5 small (77M) as Student2026.02 | 82.11 | |
| Distilling-Step-by-StepSetting=FLAN-T5 small (77M) as Student2026.02 | 81.3 | |
| Plackett-Luce RankingSetting=FLAN-T5 small (77M) as Student2026.02 | 80.47 | |
| Fine-tuningSetting=FLAN-T5 small (77M) as Student2026.02 | 78.86 | |
| TinyLLMSetting=FLAN-T5 base (248M) as Student2026.02 | 78.05 | |
| TinyLLMSetting=FLAN-T5 small (77M) as Student2026.02 | 75.61 | |
| Llama 2-chatSetting=Teacher2026.02 | 69.92 | |
| FLAN-T5 xlargeSetting=Teacher2026.02 | 65.85 | |
| InferenceSetting=FLAN-T5 large (783M) as Student2026.02 | 63.41 | |
| InferenceSetting=FLAN-T5 base (248M) as Student2026.02 | 57.72 | |
| InferenceSetting=FLAN-T5 small (77M) as Student2026.02 | 47.97 |