Long-context Inference on Memory-Constrained 8K Context Setting
7.2Time To First Token (TTFT) (s)Lazy-Attn
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Lazy-Attn2026.06 | 7.2 | 0.8 | 28.8 | 206 | |
| Block-Attention2026.06 | 15.8 | 0.42 | 9.6 | 847 |
Loading the SOTA2 catalog…
| Method | Links | ||||
|---|---|---|---|---|---|
| Lazy-Attn2026.06 | 7.2 | 0.8 | 28.8 | 206 | |
| Block-Attention2026.06 | 15.8 | 0.42 | 9.6 | 847 |