Latency Evaluation on NarrativeQA
2.1End-to-End LatencySnapKV
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| SnapKVCompression Constraint=32x2026.02 | 2.1 | — | — | |
| GMSACompression Constraint=32x2026.02 | 2.1 | 1.57 | 0.53 | |
| COMICompression Constraint=32x2026.02 | 3.27 | 2.76 | 0.5 | |
| Activation BeaconCompression Constraint=32x2026.02 | 3.93 | — | — | |
| Original PromptCompression Constraint=32x2026.02 | 7.04 | — | — | |
| LongLLMLinguaCompression Constraint=32x2026.02 | 70.95 | 68.84 | 2.12 |