Long-term Conversation Question Answering on Realtalk
49.2Multi-hop ScoreFull KV
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Full KVBackbone=LLaMA3.1-8B, KV Cache Budget (M)=Full, Prefill Protocol=block-prefill2025.09 | 49.2 | 55.9 | 48.4 | 52 | |
| EpiCacheBackbone=LLaMA3.1-8B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 44.2 | 50.9 | 46.5 | 47.5 | |
| EpiCacheBackbone=LLaMA3.1-8B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 42.3 | 50.5 | 35.2 | 46.5 | |
| SnapKVBackbone=LLaMA3.1-8B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 41.5 | 36.8 | 45.2 | 40 | |
| EpiCacheBackbone=LLaMA3.2-3B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 40 | 26.6 | 42.9 | 34.6 | |
| EpiCacheBackbone=LLaMA3.2-3B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 39.9 | 25.6 | 44.2 | 34.3 | |
| EpiCacheBackbone=LLaMA3.1-8B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 39.6 | 44.9 | 46.7 | 43 | |
| Full KVBackbone=LLaMA3.2-3B, KV Cache Budget (M)=Full, Prefill Protocol=block-prefill2025.09 | 39 | 30.8 | 38.2 | 35.3 | |
| KVzipBackbone=LLaMA3.1-8B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 38.8 | 41.3 | 41.1 | 40.3 | |
| Full KVBackbone=Qwen2.5-7B2025.09 | 38.7 | 52.3 | 43.4 | 45.3 | |
| InfiniPotBackbone=LLaMA3.1-8B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 38.3 | 38.2 | 43.4 | 39 | |
| EpiCacheBackbone=LLaMA3.1-8B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 37.9 | 35.2 | 45.2 | 37.8 | |
| EpiCacheBackbone=LLaMA3.2-3B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 37.4 | 24.7 | 41 | 32.4 | |
| SnapKVBackbone=LLaMA3.1-8B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 37.3 | 32.2 | 42.3 | 35.8 | |
| KVzipBackbone=LLaMA3.1-8B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 36.4 | 33.4 | 41.5 | 35.8 | |
| InfiniPotBackbone=LLaMA3.2-3B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 35.6 | 24.5 | 36.9 | 31 | |
| EpiCacheBackbone=LLaMA3.2-3B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 34.7 | 23 | 41.3 | 30.5 | |
| InfiniPotBackbone=LLaMA3.1-8B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 34.4 | 33.3 | 43.4 | 35.3 | |
| SnapKVBackbone=LLaMA3.2-3B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 34.1 | 20.3 | 33.7 | 28 | |
| EPICACHEBackbone=Qwen2.5-3B, M (KV Cache Size)=8K2025.09 | 33.9 | 25 | 41.1 | 31.1 | |
| EPICACHEBackbone=Qwen2.5-7B, M (KV Cache Size)=8K2025.09 | 33.8 | 46.9 | 43.6 | 41 | |
| KeyDiffBackbone=LLaMA3.1-8B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 33 | 29.9 | 33.3 | 31.7 | |
| EPICACHEBackbone=Qwen2.5-7B, M (KV Cache Size)=6K2025.09 | 33 | 39.4 | 40.6 | 36.9 | |
| InfiniPotBackbone=LLaMA3.2-3B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 32.8 | 19.7 | 32.4 | 27 | |
| Full KVBackbone=Qwen2.5-3B2025.09 | 32.7 | 28 | 39.6 | 31.6 | |
| EPICACHEBackbone=Qwen2.5-3B, M (KV Cache Size)=6K2025.09 | 32.6 | 19.7 | 36.4 | 27.5 | |
| SnapKVBackbone=LLaMA3.2-3B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 31.7 | 18.2 | 33.2 | 26 | |
| EPICACHEBackbone=Qwen2.5-7B, M (KV Cache Size)=4K2025.09 | 31.1 | 29.3 | 37.5 | 31.3 | |
| KVzipBackbone=LLaMA3.2-3B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 30.7 | 18.1 | 34.5 | 25.7 | |
| InfiniPotBackbone=LLaMA3.1-8B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 30 | 25.8 | 39.1 | 29.5 | |
| EPICACHEBackbone=Qwen2.5-3B, M (KV Cache Size)=4K2025.09 | 30 | 17.1 | 36.9 | 25.4 | |
| KeyDiffBackbone=LLaMA3.1-8B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 29.6 | 25.3 | 33.3 | 28.2 | |
| KVzipBackbone=LLaMA3.1-8B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 29.5 | 26.4 | 40 | 29.7 | |
| InfiniPotBackbone=LLaMA3.2-3B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 29.2 | 16 | 31.9 | 23.8 | |
| SnapKVBackbone=LLaMA3.1-8B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 28.7 | 26.6 | 44.4 | 30.1 | |
| SnapKVBackbone=LLaMA3.2-3B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 28.5 | 14.7 | 30 | 22.7 | |
| KVzipBackbone=Qwen2.5-7B, M (KV Cache Size)=8K2025.09 | 26.8 | 28.7 | 37.7 | 29.3 | |
| InfiniPotBackbone=Qwen2.5-7B, M (KV Cache Size)=8K2025.09 | 26.3 | 29.4 | 31.5 | 29 | |
| KVzipBackbone=LLaMA3.2-3B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 26.1 | 15.2 | 36.3 | 22.8 | |
| EPICACHEBackbone=Qwen2.5-3B, M (KV Cache Size)=2K2025.09 | 25.6 | 13.2 | 37.7 | 22 | |
| KVzipBackbone=LLaMA3.2-3B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 25 | 10.1 | 29.4 | 19.1 | |
| KeyDiffBackbone=LLaMA3.2-3B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 24.8 | 20.7 | 29.7 | 23.7 | |
| SnapKVBackbone=Qwen2.5-7B, M (KV Cache Size)=8K2025.09 | 24.4 | 25.1 | 33.5 | 26.1 | |
| EPICACHEBackbone=Qwen2.5-7B, M (KV Cache Size)=2K2025.09 | 24.1 | 21.3 | 36.6 | 24.7 | |
| InfiniPotBackbone=Qwen2.5-3B, M (KV Cache Size)=8K2025.09 | 23.9 | 11.4 | 31.1 | 19.5 | |
| KVzipBackbone=Qwen2.5-7B, M (KV Cache Size)=6K2025.09 | 22.5 | 24.6 | 32.6 | 24.9 | |
| InfiniPotBackbone=LLaMA3.2-3B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 22.1 | 9.4 | 24.3 | 16.8 | |
| KeyDiffBackbone=LLaMA3.1-8B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 21.9 | 16.5 | 30.9 | 20.9 | |
| SnapKVBackbone=Qwen2.5-3B, M (KV Cache Size)=8K2025.09 | 21.6 | 12.6 | 28.8 | 18.7 | |
| InfiniPotBackbone=Qwen2.5-7B, M (KV Cache Size)=6K2025.09 | 21.6 | 30 | 25.5 | 25.8 | |
| InfiniPotBackbone=LLaMA3.1-8B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 21.2 | 14.9 | 35 | 20.5 | |
| KeyDiffBackbone=LLaMA3.2-3B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 20.6 | 14.5 | 24.8 | 18.5 | |
| KVzipBackbone=LLaMA3.1-8B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 20.6 | 16.6 | 34.6 | 20.9 | |
| SnapKVBackbone=Qwen2.5-7B, M (KV Cache Size)=6K2025.09 | 20.6 | 24.4 | 29.2 | 23.5 | |
| KeyDiffBackbone=Qwen2.5-7B, M (KV Cache Size)=8K2025.09 | 20.3 | 32.7 | 28 | 26.9 | |
| SnapKVBackbone=LLaMA3.1-8B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 20.2 | 15.8 | 34.3 | 20.4 | |
| SnapKVBackbone=LLaMA3.2-3B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 20.1 | 10.5 | 25 | 16.6 | |
| KVzipBackbone=Qwen2.5-3B, M (KV Cache Size)=8K2025.09 | 18.8 | 11.4 | 28.4 | 17 | |
| KVzipBackbone=LLaMA3.2-3B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 18.7 | 9.5 | 27.9 | 16 | |
| KeyDiffBackbone=Qwen2.5-3B, M (KV Cache Size)=8K2025.09 | 18.1 | 15.1 | 24.4 | 17.7 | |
| SnapKVBackbone=Qwen2.5-3B, M (KV Cache Size)=6K2025.09 | 17.9 | 9.3 | 26.2 | 15.3 | |
| KVzipBackbone=Qwen2.5-7B, M (KV Cache Size)=4K2025.09 | 17.7 | 16 | 29.8 | 18.8 | |
| InfiniPotBackbone=Qwen2.5-3B, M (KV Cache Size)=6K2025.09 | 17.5 | 12 | 28.2 | 16.7 | |
| InfiniPotBackbone=Qwen2.5-7B, M (KV Cache Size)=4K2025.09 | 17.1 | 22.7 | 23.7 | 20.5 | |
| KeyDiffBackbone=LLaMA3.2-3B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 16.8 | 14.2 | 18.2 | 15.9 | |
| SnapKVBackbone=Qwen2.5-7B, M (KV Cache Size)=4K2025.09 | 15.8 | 16 | 24.8 | 17.2 | |
| KeyDiffBackbone=Qwen2.5-7B, M (KV Cache Size)=6K2025.09 | 15.7 | 26.1 | 21.1 | 21.1 | |
| SnapKVBackbone=Qwen2.5-3B, M (KV Cache Size)=4K2025.09 | 15.4 | 9 | 27 | 14.3 | |
| InfiniPotBackbone=Qwen2.5-3B, M (KV Cache Size)=4K2025.09 | 15.4 | 8.9 | 24.3 | 13.9 | |
| KVzipBackbone=Qwen2.5-3B, M (KV Cache Size)=6K2025.09 | 15 | 10 | 27.8 | 14.7 | |
| KVzipBackbone=Qwen2.5-3B, M (KV Cache Size)=4K2025.09 | 13.3 | 8.4 | 21.7 | 12.4 | |
| KeyDiffBackbone=LLaMA3.1-8B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 13 | 8.2 | 28.8 | 13.2 | |
| SnapKVBackbone=Qwen2.5-3B, M (KV Cache Size)=2K2025.09 | 12.4 | 5.6 | 23.5 | 11.1 | |
| KVzipBackbone=Qwen2.5-7B, M (KV Cache Size)=2K2025.09 | 12.2 | 10.8 | 23.9 | 13.3 | |
| KeyDiffBackbone=Qwen2.5-7B, M (KV Cache Size)=4K2025.09 | 12 | 19.9 | 26.3 | 19.5 | |
| KeyDiffBackbone=Qwen2.5-3B, M (KV Cache Size)=6K2025.09 | 11.5 | 10 | 22.6 | 12.5 | |
| KVzipBackbone=Qwen2.5-3B, M (KV Cache Size)=2K2025.09 | 10.6 | 5.3 | 18.2 | 9.4 | |
| KeyDiffBackbone=LLaMA3.2-3B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 10.4 | 5.6 | 19.5 | 9.7 | |
| InfiniPotBackbone=Qwen2.5-7B, M (KV Cache Size)=2K2025.09 | 10.4 | 15.7 | 15.2 | 13.4 | |
| SnapKVBackbone=Qwen2.5-7B, M (KV Cache Size)=2K2025.09 | 9.8 | 10.5 | 14.2 | 10.8 | |
| KeyDiffBackbone=Qwen2.5-7B, M (KV Cache Size)=2K2025.09 | 9.8 | 15 | 18.4 | 13.3 | |
| KeyDiffBackbone=Qwen2.5-3B, M (KV Cache Size)=4K2025.09 | 9.2 | 6 | 21.6 | 9.6 | |
| InfiniPotBackbone=Qwen2.5-3B, M (KV Cache Size)=2K2025.09 | 8.6 | 6.1 | 26.3 | 10.1 | |
| KeyDiffBackbone=Qwen2.5-3B, M (KV Cache Size)=2K2025.09 | 3.9 | 9 | 23.5 | 8.9 |