LLM Decoding Throughput
Benchmarks
DatasetSOTA methodMetricTrendResultsLast updated
2,312.1Throughput (tok/s)
24
Jun 4, 2026
1,358.2Throughput (tok/s)
24
Jun 4, 2026
1,996.3Throughput (tok/s)
21
Jun 4, 2026
1,159.1Throughput (tok/s)
21
Jun 4, 2026
1,694.1Throughput (tok/s)
19
Jun 4, 2026
1,076.3Decoding Throughput (tok/s)
19
Jun 4, 2026
1,463.3Throughput (tok/s)
16
Jun 4, 2026
1,000.1Decoding Throughput (tok/s)
16
Jun 4, 2026
465Throughput (tokens/s)
12
Jul 2, 2026