Long-context Question Answering on LongBench N=162
31.5F1 ScoreRef. ceiling
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Ref. ceilingCache capacity (C)=256, Backbone=Qwen2.5-3B2026.05 | 31.5 | 100 | |
| H2O-faithful+protCache capacity (C)=256, Protection=10% prefix + 10% suffix, Implementation=Faithful reproduction, Backbone=Qwen2.5-3B2026.05 | 30 | 95.4 | |
| SnapKV-faithful+protCache capacity (C)=256, Protection=10% prefix + 10% suffix, Implementation=Faithful reproduction, Backbone=Qwen2.5-3B2026.05 | 29.9 | 95.1 | |
| H2O+protCache capacity (C)=512, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 29.8 | — | |
| SnapKV+protCache capacity (C)=512, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 29.8 | — | |
| Ada-KV+protCache capacity (C)=512, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 29.8 | — | |
| QUEST+protCache capacity (C)=512, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 29.8 | — | |
| H2O+protCache capacity (C)=256, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 29 | 91.9 | |
| SnapKV+protCache capacity (C)=256, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 29 | 91.9 | |
| Ada-KV+protCache capacity (C)=256, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 29 | 91.9 | |
| QUEST+protCache capacity (C)=256, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 29 | 91.9 | |
| Random+protCache capacity (C)=512, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 28.9 | — | |
| LRU+protCache capacity (C)=512, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 28.5 | — | |
| LRU+protCache capacity (C)=256, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 28.2 | 89.4 | |
| Random+protCache capacity (C)=256, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 25 | 79.5 | |
| H2O+protCache capacity (C)=128, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 23 | — | |
| SnapKV+protCache capacity (C)=128, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 23 | — | |
| Ada-KV+protCache capacity (C)=128, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 23 | — | |
| QUEST+protCache capacity (C)=128, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 23 | — | |
| LRU+protCache capacity (C)=128, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 22.9 | — | |
| Random+protCache capacity (C)=128, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 22.5 | — | |
| SLW+protCache capacity (C)=512, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 19.8 | — | |
| SLW+protCache capacity (C)=256, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 18.4 | 58.5 | |
| SLW+protCache capacity (C)=128, Protection=10% prefix + 10% suffix, Backbone=Qwen2.5-3B2026.05 | 12.4 | — | |
| H2OCache capacity (C)=256, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 3.8 | 12.2 | |
| H2OCache capacity (C)=512, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 3.8 | — | |
| SnapKVCache capacity (C)=256, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 3.8 | 12 | |
| SnapKVCache capacity (C)=512, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 3.7 | — | |
| RandomCache capacity (C)=512, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 3.5 | — | |
| H2OCache capacity (C)=128, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 3 | — | |
| SnapKVCache capacity (C)=128, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 3 | — | |
| SLWCache capacity (C)=512, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 3 | — | |
| SLWCache capacity (C)=256, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 2.7 | 8.4 | |
| RandomCache capacity (C)=256, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 2.7 | 8.5 | |
| SLWCache capacity (C)=128, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 2.1 | — | |
| LRUCache capacity (C)=128, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 1.9 | — | |
| LRUCache capacity (C)=256, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 1.1 | 3.6 | |
| QUESTCache capacity (C)=128, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 1.1 | — | |
| LRUCache capacity (C)=512, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 1 | — | |
| RandomCache capacity (C)=128, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 0.9 | — | |
| QUESTCache capacity (C)=256, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 0.8 | 2.4 | |
| Ada-KVCache capacity (C)=256, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 0.5 | 1.6 | |
| Ada-KVCache capacity (C)=512, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 0.5 | — | |
| Ada-KVCache capacity (C)=128, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 0.4 | — | |
| QUESTCache capacity (C)=512, Protection=Without protection, Backbone=Qwen2.5-3B2026.05 | 0.4 | — |