List your company
ResearchFormerly WizWand

LLM Inference Scheduling on Alpaca

0.1576P90 Per-token Latency (s/token)Oracle SJF
-0.2904862.734165.7588058.78345Sep 25, 2025Oct 26, 2025Nov 26, 2025Dec 28, 2025Jan 28, 2026Feb 28, 2026Apr 1, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Oracle SJFModel=Llama, Workload Mode=Burst Mode2025.090.1576———
PARSModel=Llama, Workload Mode=Burst Mode2025.090.18———
Cross-Model PARSModel=Llama, Workload Mode=Burst Mode2025.090.208———
Listwise SJFModel=Llama, Workload Mode=Burst Mode2025.090.2155———
Oracle SJFModel=DeepSeek-R1, Workload Mode=Burst Mode2025.090.2298———
Pointwise SJFModel=Llama, Workload Mode=Burst Mode2025.090.2306———
PARSModel=DeepSeek-R1, Workload Mode=Burst Mode2025.090.2581———
Cross-Model PARSModel=DeepSeek-R1, Workload Mode=Burst Mode2025.090.3334———
Listwise SJFModel=DeepSeek-R1, Workload Mode=Burst Mode2025.090.3868———
Pointwise SJFModel=DeepSeek-R1, Workload Mode=Burst Mode2025.090.7052———
FCFSModel=DeepSeek-R1, Workload Mode=Burst Mode2025.090.8787———
TIETesting Model Size=8B, Request Rate (RPS)=100, Training Dataset=LMSYS-Chat-1M (8B)2026.040.930.5254.18123.02
SSJFTesting Model Size=8B, Request Rate (RPS)=100, Training Dataset=LMSYS-Chat-1M (8B)2026.041.470.7162.93140.8
LTRTesting Model Size=8B, Request Rate (RPS)=100, Training Dataset=LMSYS-Chat-1M (8B)2026.041.760.8365.33142.17
FCFSModel=Llama, Workload Mode=Burst Mode2025.092.2257———
TIETesting Model Size=70B, Request Rate (RPS)=100, Training Dataset=LMSYS-Chat-1M (8B)2026.042.811.54146.14359.08
FCFSTesting Model Size=8B, Request Rate (RPS)=100, Training Dataset=LMSYS-Chat-1M (8B)2026.043.621.4583.65159.72
SSJFTesting Model Size=70B, Request Rate (RPS)=100, Training Dataset=LMSYS-Chat-1M (8B)2026.044.322.06171.14395.78
LTRTesting Model Size=70B, Request Rate (RPS)=100, Training Dataset=LMSYS-Chat-1M (8B)2026.045.142.36174.31394.1
FCFSTesting Model Size=70B, Request Rate (RPS)=100, Training Dataset=LMSYS-Chat-1M (8B)2026.0411.364.52235.64444.52