Language Modeling Inference
Benchmarks
Dataset NameSOTA methodMetricTrendResultsLast Updated
26.3Decode Latency (ms/token)
4
Mar 24, 2026
18.4Decode Latency (ms/token)
4
Mar 24, 2026
13.6Decode Latency (ms/token)
4
Mar 24, 2026
10.3Decoding Latency (ms/token)
4
Mar 24, 2026
8.4Decode Latency (ms/token)
4
Mar 24, 2026
7.1Decode Latency (ms/token)
4
Mar 24, 2026
6.2Decode Latency (ms/token)
4
Mar 24, 2026