Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

Time to First Token (TTFT) research benchmarks · SOTA2 Research

Time to First Token (TTFT)

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

182Time to First Token (ms)

32

Jun 19, 2026

LLM Prefix-based Inference

51TTFT (ms)

12

Jun 19, 2026

Llama-3.1-8B-Instruct 128K input length

TriangleMix + MInference

14.5TTFT (s)

9

Apr 21, 2026

Llama 8B Instruct 112K input length 3.1

TriangleMix + MInference

12.7Time-to-First-Token (s)

9

Apr 21, 2026

Llama-3.1-8B-Instruct 96K input length

TriangleMix + MInference

10.9TTFT (s)

9

Apr 21, 2026

Llama-3.1-8B-Instruct 80K input length

TriangleMix + FlexPrefill

9.2TTFT (s)

9

Apr 21, 2026

Llama 8B Instruct 64K input length 3.1

TriangleMix + FlexPrefill

7.2TTFT (s)

9

Apr 21, 2026

Llama 8B Instruct 48K input length 3.1

TriangleMix + FlexPrefill

5.2TTFT (s)

9

Apr 21, 2026

Llama 8B Instruct 32K input length 3.1

TriangleMix + FlexPrefill

3.4TTFT (s)

9

Apr 21, 2026

Behavior-Equivalent Token

19.99TTFT (ms)

8

Feb 26, 2026

Behavior-Equivalent Token

19.17TTFT (ms)

8

Feb 26, 2026

Generalization Models Tulu3, Qwen3, Lego-Link0

1,195.6TTFT (Standard, ms)

3

Jun 4, 2026

Popular datasets

Llama GSM8K RoleLLM Generalization Models NVFP4

Follow for updates

Join SOTA2 Contact