Inference Efficiency
Benchmarks
Dataset NameSOTA methodMetricTrendResultsLast Updated
6.6Inference Throughput (QPS)
4
Feb 26, 2026
756,000Total Time (ms)
4
Feb 27, 2026
0.005RTF
3
Jun 8, 2026
0.58First-token Latency (s)
3
May 1, 2026
212.3Throughput (tokens/s)
3
May 1, 2026
2.3Average Rollout Time (s)
3
Mar 10, 2026
35.8Throughput (tok/s)
2
Jul 9, 2026
18.98TTFT (ms)
2
Mar 24, 2026
5,187,584Cells per Epoch
2
Mar 19, 2026