Multi-Objective Routing in LLM Serving on LMSYS-Chat-1M (Non-stationary Workload Shift)
1.47Average EELLOR
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LORWorkload=Non-stationary rate-shift, Decode lengths=noisy: Unif(0.8oj, 1.2oj)2026.07 | 1.47 | 6.72 | 9.67 | 1.48 | 3.92 | |
| Power-of-2Workload=Non-stationary rate-shift, Decode lengths=noisy: Unif(0.8oj, 1.2oj)2026.07 | 1.8 | 6.35 | 8.28 | 1.39 | 3.28 | |
| AHD Bid Price Control RouterWorkload=Non-stationary rate-shift, Decode lengths=noisy: Unif(0.8oj, 1.2oj)2026.07 | 44.81 | 45.69 | 31.15 | 1.93 | 14.57 |