Runtime efficiency on NIAH (500 samples)
34.01Latency (8K Context)H2O
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| H2OInput Length=2K tokens, Output Length=1K tokens2026.05 | 34.01 | 34.72 | 37.15 | 40.34 | 44.46 | 53.37 | |
| SnapKVInput Length=2K tokens, Output Length=1K tokens2026.05 | 34.23 | 34.86 | 37.31 | 40.56 | 44.62 | 53.78 | |
| ZeroMergeInput Length=2K tokens, Output Length=1K tokens2026.05 | 34.87 | 35.53 | 38.07 | 41.21 | 45.35 | 54.62 | |
| Judge QInput Length=2K tokens, Output Length=1K tokens2026.05 | 35.12 | 35.86 | 38.43 | 41.72 | 45.94 | 55.29 | |
| Meta-SoftInput Length=2K tokens, Output Length=1K tokens2026.05 | 35.29 | 36.03 | 38.72 | 42.05 | 46.36 | 55.78 | |
| Full KVInput Length=2K tokens, Output Length=1K tokens2026.05 | 43.27 | 54.38 | 79.93 | 142.95 | 271.83 | 583.41 |