Long-context Conversational Question Answering on LoCoMo
43.1Multi-Hop F1Full KV
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Full KVBackbone=LLaMA3.1-8B, KV Cache Budget (M)=Full, Prefill Protocol=block-prefill2025.09 | 43.1 | — | 22.7 | — | 17.2 | — | 67.4 | — | 50.5 | — | |
| Mem0Backbone=GPT-4o2026.05 | 42.57 | 30.92 | 44.55 | 32.6 | 23.04 | 17.62 | 48.49 | 37 | 39.66 | 29.54 | |
| BeliefMemBackbone=GPT-4o2026.05 | 41.24 | 35.68 | 50.81 | 45.05 | 27.6 | 23.78 | 51.81 | 43.8 | 42.87 | 37.08 | |
| BeliefMemBackbone=GPT-4o-mini2026.05 | 40.51 | 32.24 | 51.88 | 45.78 | 28.73 | 25.12 | 48.41 | 42.07 | 42.38 | 36.3 | |
| Mem0*Backbone=GPT-4o-mini, Configuration=Official reported performance2026.05 | 38.72 | 27.13 | 48.93 | 40.51 | 28.64 | 21.58 | 47.65 | 38.72 | 40.99 | 31.99 | |
| EpiCacheBackbone=LLaMA3.1-8B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 38.4 | — | 37.5 | — | 17.4 | — | 62.7 | — | 50.2 | — | |
| EpiCacheBackbone=LLaMA3.1-8B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 38.2 | — | 36.4 | — | 16.4 | — | 58.2 | — | 47.3 | — | |
| SnapKVBackbone=LLaMA3.1-8B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 37.7 | — | 18.1 | — | 14.2 | — | 55.5 | — | 41.8 | — | |
| EpiCacheBackbone=LLaMA3.1-8B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 37.7 | — | 33.8 | — | 15.9 | — | 55.8 | — | 45.4 | — | |
| KVzipBackbone=LLaMA3.1-8B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 37.2 | — | 35 | — | 13.1 | — | 54.6 | — | 44.8 | — | |
| Mem0Backbone=GPT-4o-mini2026.05 | 36.83 | 26.5 | 34.52 | 26.38 | 22.57 | 16.54 | 46.89 | 37.63 | 35.2 | 26.76 | |
| Full KVBackbone=LLaMA3.2-3B, KV Cache Budget (M)=Full, Prefill Protocol=block-prefill2025.09 | 36 | — | 15.1 | — | 13.2 | — | 54.5 | — | 40.3 | — | |
| EpiCacheBackbone=LLaMA3.2-3B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 33.9 | — | 23.9 | — | 12.7 | — | 48.2 | — | 38.3 | — | |
| EpiCacheBackbone=LLaMA3.2-3B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 33.7 | — | 22.2 | — | 12.1 | — | 46.4 | — | 36.9 | — | |
| SnapKVBackbone=LLaMA3.1-8B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 33.3 | — | 13.8 | — | 13.3 | — | 46.8 | — | 35.3 | — | |
| EpiCacheBackbone=LLaMA3.1-8B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 33.2 | — | 26.3 | — | 14.2 | — | 43.5 | — | 36.3 | — | |
| InfiniPotBackbone=LLaMA3.1-8B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 33.1 | — | 33.8 | — | 13.8 | — | 45.5 | — | 38.8 | — | |
| KeyDiffBackbone=LLaMA3.1-8B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 33 | — | 25.8 | — | 16.4 | — | 46.2 | — | 37.7 | — | |
| A-MEM*Backbone=GPT-4o, Configuration=Official reported performance2026.05 | 32.86 | 23.76 | 39.41 | 31.23 | 17.1 | 15.84 | 48.43 | 42.97 | 34.45 | 28.45 | |
| KVzipBackbone=LLaMA3.1-8B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 32.8 | — | 31 | — | 13.3 | — | 47.3 | — | 39.1 | — | |
| A-MEMBackbone=GPT-4o2026.05 | 31.66 | 23.34 | 41.11 | 34.72 | 17.45 | 15.58 | 47.04 | 41.02 | 34.32 | 28.67 | |
| SnapKVBackbone=LLaMA3.2-3B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 31.2 | — | 11.9 | — | 11.7 | — | 44.3 | — | 33.1 | — | |
| KVzipBackbone=LLaMA3.2-3B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 31.2 | — | 21.8 | — | 11.8 | — | 41.5 | — | 33.6 | — | |
| SnapKVBackbone=LLaMA3.1-8B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 30.8 | — | 10.9 | — | 13.6 | — | 37.3 | — | 29.2 | — | |
| EpiCacheBackbone=LLaMA3.2-3B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 30.4 | — | 19.7 | — | 14.2 | — | 42.3 | — | 33.6 | — | |
| MemGPTBackbone=GPT-4o2026.05 | 30.36 | 22.83 | 17.29 | 13.18 | 12.24 | 11.87 | 60.18 | 53.35 | 30.02 | 25.31 | |
| InfiniPotBackbone=LLaMA3.1-8B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 29.8 | — | 29.6 | — | 13.9 | — | 40.1 | — | 34.5 | — | |
| EpiCacheBackbone=LLaMA3.2-3B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 29.3 | — | 15.9 | — | 13.7 | — | 33.1 | — | 27.6 | — | |
| KVzipBackbone=LLaMA3.1-8B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 28.8 | — | 24.3 | — | 15.3 | — | 36.8 | — | 31.4 | — | |
| InfiniPotBackbone=LLaMA3.2-3B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 28.6 | — | 20.5 | — | 11.4 | — | 36.9 | — | 30.4 | — | |
| KVzipBackbone=LLaMA3.2-3B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 28.1 | — | 16.5 | — | 12 | — | 34.2 | — | 28 | — | |
| LoCoMoBackbone=GPT-4o2026.05 | 28 | 18.47 | 9.09 | 5.78 | 16.47 | 14.8 | 61.56 | 54.19 | 28.78 | 23.31 | |
| KeyDiffBackbone=LLaMA3.1-8B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 27.5 | — | 18.6 | — | 14.5 | — | 39.7 | — | 31.5 | — | |
| SnapKVBackbone=LLaMA3.2-3B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 27.3 | — | 9.9 | — | 10.6 | — | 36.8 | — | 27.8 | — | |
| A-MEMBackbone=GPT-4o-mini2026.05 | 27.08 | 20.46 | 29.14 | 24.08 | 16.6 | 13.8 | 40.7 | 32.63 | 28.38 | 22.74 | |
| A-MEM*Backbone=GPT-4o-mini, Configuration=Official reported performance2026.05 | 27.02 | 20.09 | 45.85 | 36.67 | 12.14 | 12 | 44.65 | 37.06 | 32.42 | 26.46 | |
| MemGPTBackbone=GPT-4o-mini2026.05 | 26.65 | 17.72 | 25.52 | 19.44 | 9.15 | 7.44 | 41.04 | 34.34 | 25.59 | 19.74 | |
| InfiniPotBackbone=LLaMA3.2-3B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 25.6 | — | 17 | — | 11.9 | — | 31.3 | — | 26.1 | — | |
| LoCoMoBackbone=GPT-4o-mini2026.05 | 25.02 | 19.75 | 18.41 | 14.77 | 12.04 | 11.16 | 40.36 | 29.05 | 23.96 | 18.68 | |
| KVzipBackbone=LLaMA3.1-8B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 24.4 | — | 8.4 | — | 22.8 | — | 24.7 | — | 21.2 | — | |
| KeyDiffBackbone=LLaMA3.2-3B, KV Cache Budget (M)=8K, Prefill Protocol=block-prefill2025.09 | 23.7 | — | 15.7 | — | 14.7 | — | 30.2 | — | 25 | — | |
| SnapKVBackbone=LLaMA3.1-8B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 23.4 | — | 6.6 | — | 14.3 | — | 25.5 | — | 20.5 | — | |
| KeyDiffBackbone=LLaMA3.1-8B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 23.3 | — | 14.1 | — | 13.8 | — | 31.5 | — | 25.3 | — | |
| SnapKVBackbone=LLaMA3.2-3B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 23.1 | — | 6 | — | 11.5 | — | 28.7 | — | 21.9 | — | |
| InfiniPotBackbone=LLaMA3.1-8B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 23 | — | 22.1 | — | 13.9 | — | 29.5 | — | 25.8 | — | |
| KVzipBackbone=LLaMA3.2-3B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 22 | — | 4.5 | — | 12 | — | 17.3 | — | 15.2 | — | |
| KVzipBackbone=LLaMA3.2-3B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 21.9 | — | 13.3 | — | 10 | — | 24.5 | — | 20.8 | — | |
| InfiniPotBackbone=LLaMA3.2-3B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 21.3 | — | 12 | — | 10 | — | 23.7 | — | 20 | — | |
| KeyDiffBackbone=LLaMA3.2-3B, KV Cache Budget (M)=6K, Prefill Protocol=block-prefill2025.09 | 20.7 | — | 11.9 | — | 13.4 | — | 23.8 | — | 20.1 | — | |
| KeyDiffBackbone=LLaMA3.1-8B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 18.7 | — | 4.4 | — | 17.7 | — | 21.8 | — | 17.3 | — | |
| SnapKVBackbone=LLaMA3.2-3B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 17.3 | — | 3.7 | — | 17.3 | — | 17.6 | — | 14.3 | — | |
| InfiniPotBackbone=LLaMA3.2-3B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 15.9 | — | 7.2 | — | 10.1 | — | 15 | — | 13.3 | — | |
| KeyDiffBackbone=LLaMA3.2-3B, KV Cache Budget (M)=4K, Prefill Protocol=block-prefill2025.09 | 15.7 | — | 8.3 | — | 11.5 | — | 17.9 | — | 15.1 | — | |
| InfiniPotBackbone=LLaMA3.1-8B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 15.2 | — | 12.3 | — | 10.2 | — | 19.6 | — | 16.7 | — | |
| ReadAgentBackbone=GPT-4o2026.05 | 14.61 | 9.95 | 4.16 | 3.19 | 8.84 | 8.37 | 12.46 | 10.29 | 10.02 | 7.95 | |
| KeyDiffBackbone=LLaMA3.2-3B, KV Cache Budget (M)=2K, Prefill Protocol=block-prefill2025.09 | 10.7 | — | 3.7 | — | 11.6 | — | 11.2 | — | 9.2 | — | |
| ReadAgentBackbone=GPT-4o-mini2026.05 | 9.15 | 6.48 | 12.6 | 8.87 | 5.31 | 5.12 | 9.67 | 7.66 | 9.18 | 7.03 | |
| MemoryBankBackbone=GPT-4o2026.05 | 6.49 | 4.69 | 2.47 | 2.43 | 6.43 | 5.3 | 8.26 | 7.1 | 5.91 | 4.88 | |
| MemoryBankBackbone=GPT-4o-mini2026.05 | 5 | 4.77 | 9.68 | 6.99 | 5.56 | 5.94 | 6.61 | 5.16 | 6.71 | 5.72 |