LLM Routing Latency Overhead on LLM Routing Systems
40Latency Overhead (ms)Hybrid LLM
Evaluation results
| Method | Links | |
|---|---|---|
| Hybrid LLMRouting Type=Pre-route, Model-Decoupled?=No, # Dims=1 (scalar), Retraining on Catalog Change?=Yes, Multilingual?=No2026.05 | 40 | |
| TRouterRouting Type=Pre-route, Model-Decoupled?=No, # Dims=1, Retraining on Catalog Change?=Yes, Multilingual?=No2026.05 | 40 | |
| RouteLLMRouting Type=Pre-route, Model-Decoupled?=No, # Dims=1 (binary), Retraining on Catalog Change?=Yes, Multilingual?=No2026.05 | 50 | |
| MTRouterRouting Type=Pre-route, Model-Decoupled?=No, # Dims=1, Retraining on Catalog Change?=Yes, Multilingual?=No2026.05 | 50 | |
| HyDRARouting Type=Pre-route, Model-Decoupled?=Yes, # Dims=4, Retraining on Catalog Change?=No, Multilingual?=Yes2026.05 | 55 |