LLM Request Scheduling on LMSYS (test)
0.1103Average Latency (s)Oracle SVF
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Oracle SVFModel=Llama-3.1-8B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 0.1103 | 0.195 | — | |
| Oracle SJFModel=Llama-3.1-8B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 0.1181 | 0.1497 | — | |
| Oracle SVFModel=Llama-3.1-70B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 0.1792 | 0.3348 | — | |
| Oracle SJFModel=Llama-3.1-70B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 0.2045 | 0.2886 | — | |
| SVFModel=Llama-3.1-8B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 0.8901 | 3.757 | 3,239.4 | |
| SJFModel=Llama-3.1-8B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 1.0087 | 4.8632 | 3,229.9 | |
| 1-bit SVFModel=Llama-3.1-8B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 1.5614 | 6.2247 | 3,361.8 | |
| SVFModel=Llama-3.1-70B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 1.609 | 9.0097 | 2,360.6 | |
| SJFModel=Llama-3.1-70B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 1.8275 | 11.4283 | 2,444.3 | |
| 1-bit SVFModel=Llama-3.1-70B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 1.985 | 6.8657 | 2,492.1 | |
| FCFSModel=Llama-3.1-8B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 2.4592 | 10.2364 | 3,338.1 | |
| FCFSModel=Llama-3.1-70B-Instruct, Arrival Scenario=Burst-arrival2026.06 | 3.0666 | 14.0373 | 2,467.9 |