Long-context Question Answering on LongMemEval LongConvQA
90.3SH ScoreInfiniPot
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| InfiniPotBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 90.3 | 9.1 | 18.8 | 54 | 28.5 | 56.8 | 9.4 | 42.2 | |
| Full KVBackbone=Qwen2.5-7B, KV Cache Capacity (M)=21K, Prefill Strategy=block prefill2025.09 | 88.6 | 39.7 | 35.1 | 32.9 | 35.4 | 47.9 | 12.9 | 46.9 | |
| EPICACHEBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 88.2 | 13.5 | 17.5 | 56.5 | 28.5 | 55.2 | 6.4 | 43 | |
| Full KVBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=21K, Prefill Strategy=block prefill2025.09 | 87.2 | 14.1 | 17.6 | 56.5 | 28.5 | 56.1 | 6.3 | 43.1 | |
| EPICACHEBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 87.2 | 14.1 | 17.4 | 56.5 | 28.5 | 54.6 | 3.9 | 42.6 | |
| EPICACHEBackbone=Qwen2.5-7B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 86.1 | 38.3 | 33.1 | 32.9 | 39.9 | 48.8 | 12.7 | 46.9 | |
| EPICACHEBackbone=Qwen2.5-7B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 86 | 38.3 | 33.1 | 37.6 | 40.5 | 47.3 | 12.9 | 47.2 | |
| EPICACHEBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 85 | 10 | 13.4 | 40.7 | 27.2 | 55.1 | 8.3 | 39.6 | |
| EPICACHEBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 85 | 10 | 12.5 | 40.7 | 26.6 | 56.6 | 6.2 | 39.5 | |
| Full KVBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=21K, Prefill Strategy=block prefill2025.09 | 84.6 | 10 | 12.5 | 47.9 | 27.1 | 52.3 | 6.2 | 39.4 | |
| EPICACHEBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 83.8 | 13.8 | 25.4 | 55.7 | 25.3 | 54.9 | 10.7 | 42.9 | |
| EPICACHEBackbone=Qwen2.5-7B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 83.6 | 38.3 | 29.8 | 37.6 | 49.6 | 41.3 | 12.8 | 46.6 | |
| InfiniPotBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 81.3 | 11 | 13.3 | 54.9 | 28.1 | 54.9 | 5.6 | 40.4 | |
| Full KVBackbone=Qwen2.5-3B, KV Cache Capacity (M)=21K, Prefill Strategy=block prefill2025.09 | 80.8 | 14 | 15 | 50.2 | 23.7 | 59 | 9.1 | 40.7 | |
| EPICACHEBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 79.9 | 12.6 | 16.6 | 41.4 | 27 | 53.9 | 9.1 | 39.3 | |
| InfiniPotBackbone=Qwen2.5-7B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 78.6 | 34 | 33.6 | 32.9 | 41.4 | 50.1 | 13.1 | 45.5 | |
| EPICACHEBackbone=Qwen2.5-3B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 77.3 | 16.7 | 15.4 | 46.7 | 22 | 55.3 | 10.5 | 39.2 | |
| EPICACHEBackbone=Qwen2.5-3B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 77.3 | 16.7 | 15 | 46.7 | 22.8 | 55.9 | 10.1 | 39.4 | |
| InfiniPotBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 76 | 4.4 | 13.3 | 40.7 | 25 | 52.9 | 7.6 | 36.2 | |
| EPICACHEBackbone=Qwen2.5-3B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 74.4 | 14 | 11 | 46.7 | 17.6 | 55.9 | 9.1 | 37 | |
| SnapKVBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 74.3 | 13.1 | 13.6 | 53.8 | 25.9 | 52.5 | 10.9 | 38.7 | |
| KVzipBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 73.6 | 14.4 | 7.7 | 48.6 | 26.9 | 51.4 | 6.2 | 37.1 | |
| KVzipBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 73.5 | 7.9 | 12.5 | 40.7 | 26.2 | 59.1 | 6.9 | 37.5 | |
| EPICACHEBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 73 | 10.5 | 7.4 | 40.5 | 21 | 50.8 | 6 | 34.4 | |
| EPICACHEBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 72.3 | 16.7 | 3.3 | 46.5 | 24 | 56.4 | 10.5 | 37.1 | |
| KeyDiffBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 70.7 | 11.2 | 18.4 | 46.1 | 30 | 50.1 | 5.2 | 37.6 | |
| EPICACHEBackbone=Qwen2.5-7B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 70.4 | 36.9 | 31.1 | 42.4 | 35.9 | 48.5 | 12.4 | 43.7 | |
| InfiniPotBackbone=Qwen2.5-3B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 70.2 | 6.1 | 15.7 | 35.9 | 27.3 | 47.3 | 7.7 | 34.5 | |
| KeyDiffBackbone=Qwen2.5-7B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 69.6 | 23.6 | 32.9 | 39.1 | 25.6 | 50.2 | 11.9 | 40.2 | |
| KVzipBackbone=Qwen2.5-7B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 69.6 | 23.5 | 23.4 | 32.9 | 35 | 40.3 | 10.6 | 37.7 | |
| KeyDiffBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 69.2 | 11.2 | 13.6 | 48.4 | 26.1 | 52.1 | 8.5 | 36.7 | |
| KVzipBackbone=Qwen2.5-3B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 68.4 | 8.8 | 15 | 39.5 | 20.7 | 40.9 | 10 | 32.5 | |
| SnapKVBackbone=Qwen2.5-7B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 67.4 | 17.1 | 24.4 | 32.9 | 31.6 | 42.9 | 14.1 | 36.7 | |
| SnapKVBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 67.1 | 7.9 | 12.4 | 37.1 | 27.1 | 56.6 | 7.1 | 35.6 | |
| InfiniPotBackbone=Qwen2.5-7B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 66.1 | 26.7 | 32.9 | 36.4 | 40.5 | 48.1 | 11.7 | 41.7 | |
| SnapKVBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 65.1 | 8.3 | 9 | 49.4 | 30.4 | 50.5 | 11.8 | 35.6 | |
| SnapKVBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 63.3 | 9.7 | 3.9 | 45.8 | 24.4 | 45.8 | 10.8 | 32.4 | |
| KeyDiffBackbone=Qwen2.5-3B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 62.6 | 22.6 | 15.4 | 30.7 | 22 | 47.5 | 7.8 | 33.9 | |
| InfiniPotBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 62.4 | 7.9 | 4.1 | 53.9 | 23 | 46.8 | 9.2 | 32.7 | |
| KVzipBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 61.3 | 11.8 | 6.9 | 42.7 | 29.4 | 47.5 | 11.3 | 33.6 | |
| InfiniPotBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 60 | 2.6 | 12.4 | 33.6 | 25.9 | 51.7 | 6.7 | 31.9 | |
| KeyDiffBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 60 | 14.7 | 12.9 | 41.4 | 20.2 | 31.8 | 6.1 | 29.6 | |
| SnapKVBackbone=Qwen2.5-3B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 59.3 | 3.9 | 19.6 | 32.4 | 5.9 | 34.5 | 5.7 | 26.4 | |
| KVzipBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 58.1 | 5.3 | 12.4 | 37.1 | 21.3 | 50.8 | 6.2 | 31.3 | |
| KVzipBackbone=Qwen2.5-7B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 57.8 | 23.5 | 22.8 | 33.6 | 31.9 | 34.4 | 11.2 | 33.8 | |
| KeyDiffBackbone=Qwen2.5-7B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 57.5 | 13 | 13.9 | 34.2 | 26.8 | 38 | 9.9 | 30.8 | |
| KVzipBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 56.6 | 9.7 | 6.7 | 35.3 | 24.1 | 42.7 | 12.4 | 29.8 | |
| SnapKVBackbone=Qwen2.5-3B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 56.3 | 11.4 | 20.4 | 29.8 | 7.9 | 41.7 | 8.1 | 28.6 | |
| KeyDiffBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=8K, Prefill Strategy=block prefill2025.09 | 56.1 | 2.1 | 6.6 | 37.1 | 25.9 | 37.7 | 7.7 | 27.9 | |
| InfiniPotBackbone=Qwen2.5-3B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 55.9 | 4.8 | 10.6 | 32.4 | 25.7 | 40.1 | 6.1 | 28.7 | |
| KeyDiffBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 55.8 | 6.5 | 7.4 | 54.3 | 11.4 | 32.2 | 9 | 26.9 | |
| SnapKVBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 54.5 | 5.3 | 15.8 | 37.1 | 23 | 58.7 | 6.3 | 33 | |
| KeyDiffBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 54.2 | 2.1 | 2.4 | 34.3 | 15.4 | 34.4 | 6.8 | 24.2 | |
| EPICACHEBackbone=Qwen2.5-3B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 52.6 | 3.5 | 10 | 32.4 | 22.9 | 46.7 | 6.7 | 28.7 | |
| KVzipBackbone=Qwen2.5-3B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 51.9 | 1.2 | 20 | 44.1 | 17.6 | 34.3 | 5.7 | 27.5 | |
| KeyDiffBackbone=Qwen2.5-3B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 51.6 | 15.3 | 21.1 | 43.7 | 20.5 | 35.4 | 9.6 | 30.4 | |
| SnapKVBackbone=Qwen2.5-7B, KV Cache Capacity (M)=6K, Prefill Strategy=block prefill2025.09 | 50.4 | 10.9 | 17 | 32.9 | 28.4 | 42.6 | 11.3 | 30.7 | |
| SnapKVBackbone=Qwen2.5-3B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 48.8 | 6.5 | 0.8 | 11.9 | 1.1 | 27.9 | 4.6 | 18.7 | |
| InfiniPotBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 48.5 | 7.9 | 12.3 | 33.6 | 18.4 | 52.3 | 7.6 | 29.4 | |
| InfiniPotBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 46.2 | 0.8 | 10.8 | 40.5 | 19.1 | 40.7 | 8.9 | 26.3 | |
| InfiniPotBackbone=Qwen2.5-3B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 46.2 | 5.3 | 16.5 | 33.3 | 19.4 | 31.5 | 6.8 | 25.2 | |
| KVzipBackbone=Qwen2.5-7B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 45.4 | 14.1 | 22 | 31.7 | 23.3 | 29.9 | 11.5 | 27.5 | |
| KVzipBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 44.7 | 2.6 | 6.5 | 37.1 | 15 | 37.8 | 7.4 | 24 | |
| InfiniPotBackbone=Qwen2.5-7B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 44.7 | 22.6 | 25.3 | 36.4 | 36.7 | 41.6 | 10.3 | 34 | |
| KeyDiffBackbone=Qwen2.5-7B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 43.1 | 5.8 | 9.9 | 37.1 | 20.5 | 32.7 | 10.7 | 24.7 | |
| SnapKVBackbone=Qwen2.5-7B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 42.9 | 10.9 | 22 | 33.6 | 28.8 | 38.4 | 10.7 | 29.3 | |
| SnapKVBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 40 | 5.3 | 2.4 | 40.2 | 20.4 | 48.5 | 6.5 | 26.1 | |
| InfiniPotBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 39.8 | 1.8 | 3.4 | 29.1 | 31.7 | 35.8 | 7.6 | 24.1 | |
| KVzipBackbone=Qwen2.5-3B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 39.4 | 6.5 | 5 | 23.8 | 13.8 | 25.3 | 6.4 | 19.2 | |
| SnapKVBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 35.6 | 3.8 | 3.3 | 29.8 | 23.3 | 25.4 | 8.8 | 20.2 | |
| KeyDiffBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 35.3 | 0.5 | 4.1 | 34.3 | 17.7 | 5.7 | 2.6 | 15.1 | |
| KVzipBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 33.8 | 7.5 | 8 | 36 | 19.4 | 27.2 | 9.1 | 21.5 | |
| InfiniPotBackbone=Qwen2.5-3B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 31 | 1.2 | 11.9 | 22.6 | 5.4 | 17.5 | 4.9 | 14.8 | |
| KVzipBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 30.8 | 0 | 1.8 | 30.9 | 15.2 | 30.3 | 7.6 | 18.2 | |
| KeyDiffBackbone=Qwen2.5-3B, KV Cache Capacity (M)=4K, Prefill Strategy=block prefill2025.09 | 30.5 | 11.4 | 11.1 | 34.5 | 18.9 | 33 | 5.7 | 22.5 | |
| InfiniPotBackbone=Qwen2.5-7B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 29.8 | 1.6 | 2 | 33.3 | 23.1 | 32.2 | 11.8 | 20.5 | |
| SnapKVBackbone=LLaMA3.2-3B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 26.9 | 0.8 | 1.8 | 26.6 | 17.4 | 31.4 | 6.1 | 17.7 | |
| KeyDiffBackbone=LLaMA3.1-8B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 26 | 6.8 | 9.5 | 28.8 | 12.3 | 22.8 | 5.5 | 17.1 | |
| SnapKVBackbone=Qwen2.5-3B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 23.5 | 1.2 | 1.3 | 11.9 | 0.1 | 25.6 | 3.8 | 12.7 | |
| KVzipBackbone=Qwen2.5-7B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 21.1 | 4.8 | 2.4 | 22.8 | 19.8 | 27.1 | 8.9 | 16.5 | |
| SnapKVBackbone=Qwen2.5-7B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 19.9 | 4.9 | 8.3 | 33.1 | 21.5 | 31.1 | 8.9 | 19.3 | |
| KVzipBackbone=Qwen2.5-3B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 18.4 | 1.2 | 1.3 | 13.7 | 10.9 | 22.5 | 4.2 | 11.8 | |
| KeyDiffBackbone=Qwen2.5-7B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 18.1 | 8.2 | 9.4 | 31.1 | 14.3 | 31.5 | 11.7 | 18.4 | |
| KeyDiffBackbone=Qwen2.5-3B, KV Cache Capacity (M)=2K, Prefill Strategy=block prefill2025.09 | 12.6 | 14 | 6.3 | 44.1 | 11.1 | 27.6 | 3.3 | 17.3 |