Prefill KV-cache memory measurement on TULU-3 (dev)
0.109Active KV-cache Memory (GiB)SPEED-24
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SPEED-24Prompt=1K, K=242026.05 | 0.109 | 22.7 | |
| SwiftKV-24Prompt=1K, K=242026.05 | 0.123 | 12.8 | |
| Full-AttnPrompt=1K2026.05 | 0.141 | 0 | |
| POP-24Prompt=1K, K=242026.05 | 0.141 | 0 | |
| SPEED-24Prompt=2K, K=242026.05 | 0.203 | 23.7 | |
| SwiftKV-24Prompt=2K, K=242026.05 | 0.232 | 12.8 | |
| Full-AttnPrompt=2K2026.05 | 0.266 | 0 | |
| POP-24Prompt=2K, K=242026.05 | 0.266 | 0 | |
| SPEED-24Prompt=4K, K=242026.05 | 0.391 | 24.2 | |
| SwiftKV-24Prompt=4K, K=242026.05 | 0.451 | 12.6 | |
| Full-AttnPrompt=4K2026.05 | 0.516 | 0 | |
| POP-24Prompt=4K, K=242026.05 | 0.516 | 0 | |
| SPEED-24Prompt=8K, K=242026.05 | 0.766 | 24.6 | |
| SwiftKV-24Prompt=8K, K=242026.05 | 0.889 | 12.5 | |
| Full-AttnPrompt=8K2026.05 | 1.016 | 0 | |
| POP-24Prompt=8K, K=242026.05 | 1.016 | 0 | |
| SPEED-24Prompt=16K, K=242026.05 | 1.516 | 24.8 | |
| SwiftKV-24Prompt=16K, K=242026.05 | 1.764 | 12.5 | |
| Full-AttnPrompt=16K2026.05 | 2.016 | 0 | |
| POP-24Prompt=16K, K=242026.05 | 2.016 | 0 | |
| SPEED-24Prompt=32K, K=242026.05 | 3.016 | 24.9 | |
| SwiftKV-24Prompt=32K, K=242026.05 | 3.514 | 12.5 | |
| Full-AttnPrompt=32K2026.05 | 4.016 | 0 | |
| POP-24Prompt=32K, K=242026.05 | 4.016 | 0 | |
| SPEED-24Prompt=64K, K=242026.05 | 6.016 | 25 | |
| SwiftKV-24Prompt=64K, K=242026.05 | 7.014 | 12.5 | |
| Full-AttnPrompt=64K2026.05 | 8.016 | 0 | |
| POP-24Prompt=64K, K=242026.05 | 8.016 | 0 | |
| SPEED-24Prompt=128K, K=242026.05 | 12.016 | 25 | |
| SwiftKV-24Prompt=128K, K=242026.05 | 14.014 | 12.5 | |
| Full-AttnPrompt=128K2026.05 | 16.016 | 0 | |
| POP-24Prompt=128K, K=242026.05 | 16.016 | 0 |