LLM Inference Efficiency on LongBench-E
3.032Prefill Time (sec)Exact
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Exact2025.02 | 3.032 | 37.769 | |
| BalanceKVcompression rate=0.25, b=256, T=22025.02 | 3.662 | 38.054 | |
| StreamingLLMcompression rate=0.252025.02 | 3.681 | 40.276 | |
| PyramidKVcompression rate=0.252025.02 | 3.748 | 37.241 | |
| SnapKVcompression rate=0.252025.02 | 3.755 | 40.426 |