LLM Prefill Throughput
Benchmarks
DatasetSOTA methodMetricTrendResultsLast updated
175,000Prefill Throughput (cuBLASLt)
114
May 12, 2026
5,128Throughput (tok/s)
48
Jul 2, 2026
5,063Prefill Throughput (tok/s)
48
Jul 2, 2026
4,796Throughput (tok/s)
48
Jul 2, 2026
4,447Prefill Throughput (tok/s)
36
Jul 2, 2026
4,748Prefill Throughput (tok/s)
36
Jul 2, 2026