Commonsense reasoning on PIQA (out-of-domain)
70.84AccuracyLlama-3.1-8B-Instruct
Evaluation results
| Method | Links | |
|---|---|---|
| Llama-3.1-8B-InstructSetting=Teacher2026.02 | 70.84 | |
| Similarity-based RouterSetting=FLAN-T5 large (783M) as Student2026.02 | 69.53 | |
| TinyLLMSetting=FLAN-T5 large (783M) as Student2026.02 | 68.88 | |
| Plackett-Luce RankingSetting=FLAN-T5 large (783M) as Student2026.02 | 68.77 | |
| PLM ClassifierSetting=FLAN-T5 large (783M) as Student2026.02 | 67.9 | |
| BioMistral-7BSetting=Teacher2026.02 | 67.46 | |
| Plackett-Luce RankingSetting=FLAN-T5 base (248M) as Student2026.02 | 63.98 | |
| Similarity-based RouterSetting=FLAN-T5 base (248M) as Student2026.02 | 63.33 | |
| Distilling-Step-by-StepSetting=FLAN-T5 large (783M) as Student2026.02 | 60.72 | |
| Llama 2-chatSetting=Teacher2026.02 | 60.5 | |
| PLM ClassifierSetting=FLAN-T5 base (248M) as Student2026.02 | 59.63 | |
| FLAN-T5 xlargeSetting=Teacher2026.02 | 58.43 | |
| Fine-tuningSetting=FLAN-T5 large (783M) as Student2026.02 | 58.43 | |
| TinyLLMSetting=FLAN-T5 base (248M) as Student2026.02 | 56.8 | |
| Distilling-Step-by-StepSetting=FLAN-T5 base (248M) as Student2026.02 | 55.93 | |
| Similarity-based RouterSetting=FLAN-T5 small (77M) as Student2026.02 | 53.97 | |
| Plackett-Luce RankingSetting=FLAN-T5 small (77M) as Student2026.02 | 52.77 | |
| InferenceSetting=FLAN-T5 large (783M) as Student2026.02 | 51.9 | |
| PLM ClassifierSetting=FLAN-T5 small (77M) as Student2026.02 | 50.05 | |
| TinyLLMSetting=FLAN-T5 small (77M) as Student2026.02 | 49.84 | |
| Distilling-Step-by-StepSetting=FLAN-T5 small (77M) as Student2026.02 | 49.29 | |
| Fine-tuningSetting=FLAN-T5 base (248M) as Student2026.02 | 47.55 | |
| Fine-tuningSetting=FLAN-T5 small (77M) as Student2026.02 | 42.33 | |
| InferenceSetting=FLAN-T5 base (248M) as Student2026.02 | 30.47 | |
| InferenceSetting=FLAN-T5 small (77M) as Student2026.02 | 20.78 |