LLM Inference Scheduling
Benchmarks
Dataset NameSOTA methodMetricTrendResultsLast Updated
0.158P90 Per-token Latency (s/token)
20
Jun 29, 2026
0.143Average Per-token Latency (s/token)
16
Jun 29, 2026
-19.1Mean End-to-End Latency
5
Jun 18, 2026
Loading the SOTA2 catalog…