Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

LLM Inference Efficiency research benchmarks · SOTA2 Research

LLM Inference Efficiency

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

Long sequence prompts

1.05TTFT (s)

24

May 26, 2026

Sequence prompts Medium

0.37TTFT (s)

24

May 26, 2026

Short sequence prompts

0.2TTFT (s)

24

May 26, 2026

40.6Time To First Token (TTFT)

12

Mar 17, 2026

Uniform Sampled Documents

191.7TTFT (ms)

9

Jun 4, 2026

HarmBench-Standard and AdvBench

2Slowdown

8

Jun 11, 2026

Instruction-in-Wild self-instructed prompts 36

Ground Truth Preditor

2.52Throughput (samples/s)

7

Feb 27, 2026

3.032Prefill Time (sec)

5

Mar 25, 2026

43.36Latency (s)

4

May 19, 2026

Synthetic LLM Workload Input 4096 Output 4096

160.32Latency (s)

4

May 11, 2026

Efficient Qwen Competition Scenarios Short (64/128), Medium (2048/256), Long (8192/256)

AFM-k5984d3s (Ours)

6.978Average Speedup

2

Jul 7, 2026

5,233.11TTFT (ms)

2

May 27, 2026

Synthetic LLM Workload Input 8192 Output 4096

162.78Latency (s)

2

May 11, 2026

Self-instructed prompts (test)

—

—Throughput (samples/s)

0

Feb 26, 2026

Popular datasets

Synthetic LLM Workload AIME HumanEval LongBench Instruction-in-Wild

Follow for updates

Join SOTA2 Contact