Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

Language Model Inference research benchmarks · SOTA2 Research

Language Model Inference

Benchmarks

Dataset NameSOTA MethodSort

Dataset NameSOTA methodMetricTrendResultsLast Updated

Mobile and IoT Device Latency Benchmark llama.cpp (inference evaluation)

4,215Throughput (Sample, tokens/s)

16

Feb 26, 2026

Llama-2 7B-Chat

76Latency (ms/token)

4

Mar 27, 2026

Sequence Bucket Ultra-long

77.97Latency (ms)

3

Apr 14, 2026

Sequence Bucket Long

69.58Latency (ms)

3

Apr 14, 2026

Sequence Bucket Medium

52.88Latency (ms)

3

Apr 14, 2026

Sequence Bucket Short

39.26Latency (ms)

3

Apr 14, 2026

stories100m 110M parameters

PyTorch (Accelerate)

298.7Tokens/s

3

Feb 26, 2026

vLLM benchmark 1024 prompts

Fisher-IntDim-E

1.839Latency (s)

2

Jun 5, 2026

Popular datasets

Sequence Bucket Llama STORIES Mobile and IoT Device Latency Benchmark vLLM benchmark

Follow for updates

Join SOTA2 Contact