Long-sequence generation on PG19 synthetic prompts
80Max Batch SizeSnapKV
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SnapKVGeneration Length=10K, Input Length=256 tokens2026.05 | 80 | 413.28 | |
| Judge QGeneration Length=10K, Input Length=256 tokens2026.05 | 80 | 398.15 | |
| Meta-SoftGeneration Length=10K, Input Length=256 tokens2026.05 | 80 | 390.17 | |
| Full cacheGeneration Length=10K, Input Length=256 tokens2026.05 | 28 | 180.49 |