LLM Inference Scheduling on Reasoning and Azure traces
-19.1Mean End-to-End LatencyUniBoost
Evaluation Results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| UniBoostScheduler=UniBoost2026.06 | -19.1 | 1.1 | 35.1 | 52.1 | 97.4 | 34 | -25.3 | -8.1 | 33.8 | 1.2 | |
| DistBoostScheduler=DistBoost2026.06 | -14 | 6 | 37.1 | -22.8 | -35 | -3.4 | -14.1 | -10.2 | 18.3 | 1.1 | |
| SarathiScheduler=Sarathi2026.06 | -12.3 | -13.2 | -7.6 | -49.9 | -40.5 | -8.1 | -6.8 | -8.4 | -7.9 | -10.9 | |
| SJFScheduler=SJF2026.06 | -11.1 | -12.5 | 11.2 | -74.6 | -70.5 | -9.7 | -5.2 | -7.1 | -6.3 | -7.9 | |
| TRAIL+Scheduler=TRAIL+2026.06 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |