Long-context Understanding on LongBench 64K context
63.8Average ScoreFull KV Cache
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Full KV CacheContext Length=64K2026.07 | 63.8 | 1 | 42.1 | 6.84 | 0 | 0 | |
| DepthWeave-KVContext Length=64K2026.07 | 62.9 | 8.3 | 72.8 | 4.89 | 0.09 | 0.027 | |
| TailorKVContext Length=64K2026.07 | 61.4 | 6.5 | 65 | 5.36 | 0.18 | 0.049 | |
| ChunkKVContext Length=64K2026.07 | 60.7 | 6 | 62.6 | 5.52 | 0.21 | 0.055 | |
| Eigen AttentionContext Length=64K2026.07 | 60.2 | 6.2 | 63.7 | 5.41 | 0.23 | 0.058 | |
| MiniCacheContext Length=64K2026.07 | 59.8 | 6.4 | 63.2 | 5.48 | 0.24 | 0.061 | |
| KVSharerContext Length=64K2026.07 | 59.5 | 6.7 | 64.1 | 5.44 | 0.26 | 0.064 | |
| PyramidKVContext Length=64K2026.07 | 59 | 6.1 | 61.8 | 5.66 | 0.28 | 0.069 | |
| SnapKVContext Length=64K2026.07 | 58.3 | 5.7 | 60.4 | 5.73 | 0.31 | 0.075 | |
| H2OContext Length=64K2026.07 | 57.1 | 5.4 | 56.9 | 5.98 | 0.36 | 0.083 | |
| StreamingLLMContext Length=64K2026.07 | 55.4 | 5.8 | 58.7 | 5.91 | 0.42 | 0.091 |