Cost-aware LLM Routing on Ollama-cloud trio 300 unique tasks x 3 repeats (hard-benchmark)
91.2Quality Score (q)Oracle (per-task max)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Oracle (per-task max)selection=optimal per-task2026.05 | 91.2 | 0.0067 | 165.4 | 5.75 | 31.4 | — | 8.8 | |
| Feasible (semknn LOO, in-samp.)protocol=Leave-one-out (LOO), data_scope=in-sample2026.05 | 87.7 | 0.0074 | 145.5 | 6.48 | 26.5 | — | 5.3 | |
| Semknn routerλ=3, cross-validation=5-fold stratified2026.05 | 87.4 | 0.007 | 160.2 | 6.2 | 26 | 5.6 | 5 | |
| Strongest in-sample fixed: fountaintype=fixed technique, data_scope=in-sample2026.05 | 82.4 | 0.0069 | 119.2 | 6.23 | 18.8 | 0.6 | — | |
| Semknn routerλ=38, cross-validation=5-fold stratified2026.05 | 82 | 0.0029 | 388.7 | 2.71 | 18.2 | 0.1 | 0.4 | |
| Cross-validated fixed-best (FB-CV)cross-validation=5-fold stratified2026.05 | 81.8 | 0.007 | 143.6 | 6.13 | 18 | — | 0.6 | |
| Always-baseline (uncoded)coding=uncoded2026.05 | 69.4 | 0.0012 | 577.9 | 1 | 0 | 12.5 | 13 |