Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

LLM Inference Performance research benchmarks · SOTA2 Research

LLM Inference Performance

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

Synthetic Poisson Process Requests

18.7TPOT Speedup (%)

28

Mar 18, 2026

56.03TTFT (ms)

12

Mar 10, 2026

LLM Inference Service Benchmark

256Max Concurrency

8

Jun 5, 2026

Context Length 200K

10.7Prefill Time (s)

3

Mar 13, 2026

Context Length 120K

5.66Prefill Time (s)

3

Mar 13, 2026

Context Length 60K

2.59Prefill Time (s)

3

Mar 13, 2026

Context Length 10K

0.45Prefill Time (s)

3

Mar 13, 2026

Popular datasets

Context Length Llama-3 Synthetic Poisson Process Requests LLM Inference Service Benchmark

Follow for updates

Join SOTA2 Contact