Long-Context LLM Inference Prefill Performance
0.62Prefill Latency (ms)Kascade
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| KascadeSeqlen=8192, Topk%=10%, Batch size=12025.12 | 0.62 | — | 1.23 | 1.62 | |
| FA3Seqlen=8192, Topk%=10%, Batch size=12025.12 | 0.76 | — | — | — | |
| Tilelang (TL)Seqlen=8192, Topk%=10%, Batch size=12025.12 | 1 | — | — | — | |
| KascadeSeqlen=262144, Topk%=10%, Batch size=12025.12 | 408.3 | — | 2.12 | 2.57 | |
| KascadeSeqlen=262144, Topk%=30%, Batch size=12025.12 | 727.55 | — | 1.19 | 1.44 | |
| AnchorSeqlen=262144, Topk%=30%, Batch size=12025.12 | 2,192.39 | 2.09 | — | — |