LLM Query Routing on SQuAD, HellaSwag, and HeadQA (out-of-domain)
89AccuracyOracle
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| OracleCategory=Topline, alpha=0.52025.10 | 89 | 0.33 | 0.72 | |
| Largest LLMCategory=Naive Baselines, alpha=0.52025.10 | 74 | 0.9 | 0.29 | |
| DiSRouter (SFT)Optimization=SFT, alpha=0.52025.10 | 74 | 0.62 | 0.43 | |
| DiSRouter (+ RL)Optimization=+ RL, alpha=0.52025.10 | 69 | 0.43 | 0.48 | |
| GraphRouterCategory=Router Baselines, alpha=0.52025.10 | 64 | 0.49 | 0.39 | |
| FORCCategory=Router Baselines, alpha=0.52025.10 | 63 | 0.53 | 0.36 | |
| RandomCategory=Naive Baselines, alpha=0.52025.10 | 59 | 0.46 | 0.36 | |
| FrugalGPTCategory=Router Baselines, alpha=0.52025.10 | 55 | 0.25 | 0.42 | |
| AutomixCategory=Router Baselines, alpha=0.52025.10 | 55 | 0.6 | 0.25 | |
| RouteLLMCategory=Router Baselines, alpha=0.52025.10 | 42 | 0.22 | 0.31 | |
| Smallest LLMCategory=Naive Baselines, alpha=0.52025.10 | 36 | 0.1 | 0.31 |