Reasoning and Multitask Language Understanding on OBQA, ARC, Riddle, PQA, and MMLU
77OBQA AccuracySimilarity-based Router
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Similarity-based RouterStudent Model=FLAN-T5 large, Teacher LLMs count=62026.02 | 77 | 60.17 | 70.78 | 62.75 | 55.26 | 65.19 | |
| Distilling-Step-by-StepStudent Model=FLAN-T5 large, Teacher LLMs count=62026.02 | 71.6 | 60.77 | 68.43 | 51 | 51.84 | 60.73 | |
| TinyLLMStudent Model=FLAN-T5 large, Teacher LLMs count=62026.02 | 70.4 | 54.25 | 67.25 | 52.75 | 49.28 | 58.79 | |
| InferenceStudent Model=FLAN-T5 large2026.02 | 50.4 | 51.07 | 39.8 | 45.5 | 45.1 | 46.37 |