Decoding
Benchmarks
DatasetSOTA methodMetricTrendResultsLast updated
1,410.39Throughput
21
Feb 26, 2026
468.7Throughput (tokens/sec)
20
May 12, 2026
426Throughput (tokens/sec)
20
May 12, 2026
359.3Throughput (tokens/sec)
20
May 12, 2026
394.81MT Throughput (token/s)
6
Feb 26, 2026
0.3RMSE ([0, 10^2])
5
Mar 6, 2026
669.1Throughput (MiB/s)
4
Mar 25, 2026
640.5Throughput (MiB/s)
4
Mar 25, 2026
116.333Latency (us/decode)
2
Jun 2, 2026
56.065Latency (us/decode)
2
Jun 2, 2026
39.173Latency (us/decode)
2
Jun 2, 2026