Context
Benchmarks
TaskDatasetSOTA resultTrendResults
120K context
62.9Attention Latency (120K Context, µs)
20
Context59
49.4mIoU
11
128K context
7,756.8Prefill Throughput (tok/s)
10
96K context
7,959.4Prefill Throughput (tok/s)
10
32K context
9,576.2Prefill Throughput (tok/s)
10
128K-context
101TTFT
8