Inference Efficiency
Benchmarks
Dataset NameSOTA methodMetricTrendResultsLast Updated
20TTFT (ms)
8
Apr 28, 2026
802.65Latency (ms)
8
Feb 26, 2026
58.21Latency (ms)
7
Jun 5, 2026
63.41Latency (ms)
7
Jun 5, 2026
58.83Latency (ms)
7
Jun 5, 2026
50.22Latency (ms)
7
Jun 5, 2026
161,312Throughput
7
May 12, 2026
4.12Time To First Token (s)
7
Mar 17, 2026
1,966SKV Count
7
Feb 26, 2026
12.65Avg k
6
Jun 26, 2026
7.998Inference Time (s)
6
Feb 26, 2026
1Total Time
5
Jun 11, 2026
27.29Inference Time (s / clip)
5
May 19, 2026
0.019Latency (s)
5
May 8, 2026
0.006Latency (Sec/Image)
5
Apr 23, 2026
1,115Theoretical Compute (TFLOPs)
5
Mar 12, 2026
9.34Avg. TPF
4
Jun 30, 2026
80.8Throughput (Hz)
4
Jun 29, 2026
0.091Speed Ratio
4
Jun 9, 2026
0.076Time per Step (sec)
4
May 18, 2026
102.44Latency (ms/tok)
4
Apr 28, 2026
58.77Action Expert Latency (ms)
4
Mar 17, 2026
263.8GFLOPS
4
Feb 26, 2026
8.9Throughput (queries/s)
4
Feb 26, 2026
15.8Inference Throughput (QPS)
4
Feb 26, 2026