LLM Request Scheduling on LongBench (test)
52.5263Average LatencyOracle SVF
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Oracle SVFModel=Llama-3.1-8B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 52.5263 | 128.7662 | — | |
| SVFModel=Llama-3.1-8B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 70.0169 | 215.1756 | 28.2 | |
| 1-bit SVFModel=Llama-3.1-8B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 72.5786 | 249.235 | 28.7 | |
| Oracle SJFModel=Llama-3.1-8B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 74.301 | 134.4113 | — | |
| SJFModel=Llama-3.1-8B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 106.1588 | 300.2542 | 25.9 | |
| Oracle SVFModel=Llama-3.1-70B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 108.4169 | 251.4195 | — | |
| FCFSModel=Llama-3.1-8B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 125.8883 | 393.1413 | 23.9 | |
| SVFModel=Llama-3.1-70B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 140.4193 | 420.2404 | 12.5 | |
| 1-bit SVFModel=Llama-3.1-70B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 141.4835 | 524.7749 | 12.4 | |
| Oracle SJFModel=Llama-3.1-70B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 144.3262 | 242.7121 | — | |
| SJFModel=Llama-3.1-70B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 173.9461 | 442.172 | 11.9 | |
| FCFSModel=Llama-3.1-70B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 226.2472 | 721.3823 | 10.9 |