LLM Routing on MMLU/CMMLU (In-distribution)
80.69PerformanceNIRT-Router
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| NIRT-Routeralpha=0.8, beta=0.22025.06 | 80.69 | 0.55 | 63.7 | |
| MIRT-Routeralpha=0.8, beta=0.22025.06 | 80.67 | 0.42 | 63.89 | |
| MIRT-Routeralpha=0.5, beta=0.52025.06 | 80.65 | 0.42 | 38.69 | |
| NIRT-Routeralpha=0.5, beta=0.52025.06 | 80.41 | 0.43 | 38.53 | |
| MIRT-Routeralpha=0.2, beta=0.82025.06 | 80.03 | 0.39 | 13.59 | |
| RouterBenchalpha=0.8, beta=0.22025.06 | 80.01 | 1.15 | 62.23 | |
| RouterBenchalpha=0.5, beta=0.52025.06 | 79.48 | 0.53 | 37.67 | |
| NIRT-Routeralpha=0.2, beta=0.82025.06 | 79.37 | 0.41 | 13.37 | |
| RouterBenchalpha=0.2, beta=0.82025.06 | 78.36 | 0.37 | 13.35 | |
| Large LLMalpha=0.8, beta=0.22025.06 | 77.53 | 12.93 | 42.02 | |
| Large LLMalpha=0.5, beta=0.52025.06 | 77.53 | 12.93 | -11.24 | |
| Large LLMalpha=0.2, beta=0.82025.06 | 77.53 | 12.93 | -64.49 | |
| RouteLLMalpha=0.8, beta=0.22025.06 | 77.25 | 12.8 | 42 | |
| RouteLLMalpha=0.5, beta=0.52025.06 | 73.59 | 11.15 | -6.32 | |
| RouteLLMalpha=0.2, beta=0.82025.06 | 66.24 | 7.51 | -33.17 | |
| HybridLLMalpha=0.2, beta=0.82025.06 | 56.65 | 2.78 | -5.84 | |
| HybridLLMalpha=0.8, beta=0.22025.06 | 54.37 | 1.98 | 40.43 | |
| HybridLLMalpha=0.5, beta=0.52025.06 | 52.42 | 1.54 | 20.27 | |
| Small LLMalpha=0.8, beta=0.22025.06 | 48.7 | 0.31 | 38.48 | |
| Small LLMalpha=0.5, beta=0.52025.06 | 48.7 | 0.31 | 23.14 | |
| Small LLMalpha=0.2, beta=0.82025.06 | 48.7 | 0.31 | 7.81 |