Language Model Inference
Benchmarks
DatasetSOTA methodMetricTrendResultsLast updated
4,215Throughput (Sample, tokens/s)
16
Feb 26, 2026
76Latency (ms/token)
4
Mar 27, 2026
77.97Latency (ms)
3
Apr 14, 2026
69.58Latency (ms)
3
Apr 14, 2026
52.88Latency (ms)
3
Apr 14, 2026
39.26Latency (ms)
3
Apr 14, 2026
298.7Tokens/s
3
Feb 26, 2026
1.839Latency (s)
2
Jun 5, 2026