Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

LLM Inference Latency research benchmarks · SOTA2 Research

LLM Inference Latency

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

256 tokens generation (test)

All (Predictor, Windowing, Bundling)

26I/O Latency (ms)

21

Feb 26, 2026

Multi-node InfiniBand cluster (2x8-GPU servers) inference performance (test)

0.939Time To First Token (TTFT) Mean (s)

4

May 8, 2026

Popular datasets

tokens generation Multi-node InfiniBand cluster

Follow for updates

Join SOTA2 Contact