LLM Routing on Multilingual production VS Code Chat telemetry (90 days) (5,001-query eval set)
100Routing Success RateOracle Routing
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Oracle RoutingDescription=Per-query upper bound2026.05 | 100 | 28 | 0 | |
| GPT-5.3 CodexRole=always-strong ceiling2026.05 | 97.3 | 0 | 67.9 | |
| HyDRA-MultiStrategy=conservative, τ=0.302026.05 | 95.2 | 17.1 | 40.2 | |
| Binary routerDescription=Prior in-house production system, threshold=0.392026.05 | 92.3 | 35.9 | 37 | |
| HyDRA-MultiStrategy=aggressive, τ=0.502026.05 | 92.3 | 38 | 33.8 | |
| GPT-5.4-miniRole=always-cheap floor2026.05 | 89.3 | 63.8 | 32.1 |