Prefill on Stage-aware Prefill
63.29TTFT (ms)SPEED-24
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SPEED-24Prompt=1K, K=242026.05 | 63.29 | 1.23 | |
| POP-24Prompt=1K, K=242026.05 | 72.67 | 1.07 | |
| SwiftKV-24Prompt=1K, K=242026.05 | 74.36 | 1.04 | |
| Full-AttnPrompt=1K, K=242026.05 | 77.64 | 1 | |
| SPEED-24Prompt=2K, K=242026.05 | 93.95 | 1.26 | |
| SwiftKV-24Prompt=2K, K=242026.05 | 102.26 | 1.16 | |
| POP-24Prompt=2K, K=242026.05 | 107.42 | 1.1 | |
| Full-AttnPrompt=2K, K=242026.05 | 118.34 | 1 | |
| SPEED-24Prompt=4K, K=242026.05 | 180.8 | 1.31 | |
| POP-24Prompt=4K, K=242026.05 | 187.92 | 1.26 | |
| SwiftKV-24Prompt=4K, K=242026.05 | 189.22 | 1.25 | |
| Full-AttnPrompt=4K, K=242026.05 | 237.35 | 1 | |
| SwiftKV-24Prompt=8K, K=242026.05 | 398.67 | 1.33 | |
| SPEED-24Prompt=8K, K=242026.05 | 404.41 | 1.31 | |
| POP-24Prompt=8K, K=242026.05 | 407.13 | 1.3 | |
| Full-AttnPrompt=8K, K=242026.05 | 530.77 | 1 | |
| SwiftKV-24Prompt=16K, K=242026.05 | 899.59 | 1.34 | |
| SPEED-24Prompt=16K, K=242026.05 | 918.59 | 1.31 | |
| POP-24Prompt=16K, K=242026.05 | 921.62 | 1.31 | |
| Full-AttnPrompt=16K, K=242026.05 | 1,206.27 | 1 | |
| SwiftKV-24Prompt=32K, K=242026.05 | 2,145.1 | 1.35 | |
| POP-24Prompt=32K, K=242026.05 | 2,182.87 | 1.33 | |
| SPEED-24Prompt=32K, K=242026.05 | 2,197.52 | 1.32 | |
| Full-AttnPrompt=32K, K=242026.05 | 2,896.03 | 1 | |
| SwiftKV-24Prompt=64K, K=242026.05 | 5,690 | 1.35 | |
| POP-24Prompt=64K, K=242026.05 | 5,757.18 | 1.33 | |
| SPEED-24Prompt=64K, K=242026.05 | 5,788.21 | 1.33 | |
| Full-AttnPrompt=64K, K=242026.05 | 7,674.12 | 1 | |
| SwiftKV-24Prompt=128K, K=242026.05 | 17,019.02 | 1.35 | |
| POP-24Prompt=128K, K=242026.05 | 17,093.09 | 1.34 | |
| SPEED-24Prompt=128K, K=242026.05 | 17,199.9 | 1.33 | |
| Full-AttnPrompt=128K, K=242026.05 | 22,898.6 | 1 |