Long-context Question Answering on LoCoMo
55.32F1 (Multi Hop)Nous
Evaluation results
| Method | Links | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| NousLLM Backbone=GPT-4o-mini2026.06 | 55.32 | — | 58.57 | — | 62.5 | — | 63.5 | — | — | — | — | — | — | — | — | |
| Block-Dist - FullBackbone=Llama-3.1-8B-Instruct, Attention Mode=Full Attention2026.05 | 50.25 | — | 17.82 | — | 67.72 | — | 36.57 | — | 17.23 | — | 37.92 | — | — | — | — | |
| Original - FullBackbone=Llama-3.1-8B-Instruct, Attention Mode=Full Attention2026.05 | 49.76 | — | 18.65 | — | 66.15 | — | 38.67 | — | 16.82 | — | 38.01 | — | — | — | — | |
| Block-Dist - BlockBackbone=Llama-3.1-8B-Instruct, Attention Mode=Block Attention2026.05 | 48.55 | — | 19.79 | — | 65.22 | — | 36.82 | — | 20.73 | — | 38.22 | — | — | — | — | |
| MemCoTModel=Qwen2.5-14B2026.04 | 45.1 | — | 57.49 | — | 38.04 | — | 63.09 | — | — | — | 57.06 | — | — | — | — | |
| QRRankerLLM=GPT-5-mini, Tokens=8542026.02 | 44.73 | — | 64.53 | — | 31.04 | — | 61.78 | — | — | — | 57.32 | — | — | — | — | |
| SimpleMemLLM=GPT-4.1-mini, Tokens=5312026.02 | 43.46 | — | 58.62 | — | 19.76 | — | 51.12 | — | — | — | 43.24 | — | — | — | — | |
| QRRankerLLM=GPT-4o-mini, Tokens=8542026.02 | 43.06 | — | 61.9 | — | 29.79 | — | 62.95 | — | — | — | 57.03 | — | — | — | — | |
| CompassMemModel=Qwen2.5-14B2026.04 | 42.32 | — | 47.18 | — | 25.88 | — | 61.02 | — | — | — | 52.52 | — | — | — | — | |
| Original - FullBackbone=Qwen3-8B, Attention Mode=Full Attention2026.05 | 40.44 | — | 18.28 | — | 58.3 | — | 32.43 | — | 23.32 | — | 34.55 | — | — | — | — | |
| MemCoTModel=GPT-4o-mini2026.04 | 40.43 | — | 60.31 | — | 42.67 | — | 64.81 | — | — | — | 58.03 | — | — | — | — | |
| Original - FullBackbone=Qwen3-4B-Instruct-2507, Attention Mode=Full Attention2026.05 | 39.98 | — | 9.6 | — | 48.83 | — | 33.38 | — | 34.75 | — | 33.31 | — | — | — | — | |
| Block-Dist - FullBackbone=Qwen3-14B, Attention Mode=Full Attention2026.05 | 39.93 | — | 13.06 | — | 61.1 | — | 34.93 | — | 36.61 | — | 37.13 | — | — | — | — | |
| MemboxLLM=GPT-4o-mini, Tokens=2,1662026.02 | 39.88 | — | 58.03 | — | 27.96 | — | 60.09 | — | — | — | 53.1 | — | — | — | — | |
| Block-Dist - FullBackbone=Qwen3-8B, Attention Mode=Full Attention2026.05 | 39.08 | — | 19.87 | — | 56.28 | — | 34.03 | — | 23.99 | — | 34.65 | — | — | — | — | |
| Original - FullBackbone=Qwen3-14B, Attention Mode=Full Attention2026.05 | 38.95 | — | 14.26 | — | 60.45 | — | 33.16 | — | 37.44 | — | 36.85 | — | — | — | — | |
| Block-Dist - FullBackbone=Qwen3-4B-Instruct-2507, Attention Mode=Full Attention2026.05 | 38.86 | — | 9.93 | — | 49.34 | — | 34.09 | — | 32.34 | — | 32.91 | — | — | — | — | |
| CompassMemLLM=GPT-4o-mini, Tokens=20,0002026.02 | 38.84 | — | 57.96 | — | 26.61 | — | 57.36 | — | — | — | 52.18 | — | — | — | — | |
| CompassMemModel=GPT-4o-mini2026.04 | 38.84 | — | 57.96 | — | 26.61 | — | 57.36 | — | — | — | 52.18 | — | — | — | — | |
| Block-Dist - BlockBackbone=Qwen3-8B, Attention Mode=Block Attention2026.05 | 38.76 | — | 16.26 | — | 57.38 | — | 32.71 | — | 22.49 | — | 33.52 | — | — | — | — | |
| Mem0LLM=GPT-4o-mini, Tokens=1,7642026.02 | 38.72 | — | 48.93 | — | 28.64 | — | 47.65 | — | — | — | 45.09 | — | — | — | — | |
| Mem0Model=GPT-4o-mini2026.04 | 38.72 | — | 48.93 | — | 28.64 | — | 47.65 | — | — | — | 45.1 | — | — | — | — | |
| MemoryOSModel=Qwen2.5-14B2026.04 | 38.19 | — | 32.24 | — | 20.27 | — | 46.33 | — | — | — | 40.28 | — | — | — | — | |
| Block-Dist - BlockBackbone=Qwen3-14B, Attention Mode=Block Attention2026.05 | 37.95 | — | 14.73 | — | 59.89 | — | 33.82 | — | 37.3 | — | 36.74 | — | — | — | — | |
| SFT-Emb-8BLLM=GPT-4o-mini, Tokens=8412026.02 | 37.06 | — | 56.27 | — | 29.11 | — | 57.22 | — | — | — | 51.58 | — | — | — | — | |
| Block-Dist - BlockBackbone=Qwen3-4B-Instruct-2507, Attention Mode=Block Attention2026.05 | 37.05 | — | 12.6 | — | 48.57 | — | 32.67 | — | 32.02 | — | 32.58 | — | — | — | — | |
| ES-MemLLM=GPT-4o-mini, Tokens=2,9252026.02 | 36.52 | — | 47.9 | — | 24.77 | — | 50.07 | — | — | — | 45.56 | — | — | — | — | |
| Full KVBackbone=Qwen2.5-7B2025.09 | 36.2 | — | 19.2 | — | 16.6 | — | 59.3 | — | — | — | 44.1 | — | — | — | — | |
| MEMFLYModel=GPT-4o2026.02 | 35.89 | 29.24 | 39.78 | 27.12 | 25.74 | 19.53 | 49.08 | 43.05 | 48.24 | 48.92 | 44.39 | 38.7 | — | — | — | |
| ZepLLM=GPT-4o-mini, Tokens=3,9112026.02 | 35.74 | — | 42 | — | 19.37 | — | 49.56 | — | — | — | 43.56 | — | — | — | — | |
| SynapseLLM=GPT-4o-mini, Tokens=8142026.02 | 35.7 | — | 50.1 | — | 25.9 | — | 48.9 | — | — | — | 40.5 | — | — | — | — | |
| MemoryOSLLM=GPT-4o-mini, Tokens=3,8742026.02 | 35.27 | — | 41.15 | — | 20.02 | — | 48.62 | — | — | — | 42.84 | — | — | — | — | |
| MemoryOSModel=GPT-4o-mini2026.04 | 35.27 | — | 41.15 | — | 20.02 | — | 48.62 | — | — | — | 42.84 | — | — | — | — | |
| Qwen3-Emb-8B (out-of-box)LLM=GPT-4o-mini, Tokens=8462026.02 | 35.24 | — | 41.36 | — | 24.79 | — | 47.95 | — | — | — | 42.81 | — | — | — | — | |
| MEM-0Model=GPT-4o2026.02 | 35.13 | 27.56 | 52.38 | 44.15 | 17.73 | 15.92 | 39.12 | 35.43 | 25.44 | 24.19 | 36.59 | 32.25 | — | — | — | |
| MEM-0Model=GPT-4o-mini2026.02 | 34.72 | 25.13 | 45.93 | 35.51 | 22.64 | 15.58 | 43.65 | 37.42 | 30.15 | 27.44 | 38.7 | 32.07 | — | — | — | |
| LightMemModel=GPT-4o, Token Length=6802026.04 | 34.5 | 25.1 | 40.1 | 32 | 18.4 | 16.9 | 49.8 | 44.2 | 41.5 | 39.8 | — | — | — | — | — | |
| CAMModel=Qwen2.5-14B2026.04 | 34.5 | — | 44.7 | — | 23.86 | — | 50.39 | — | — | — | 44.64 | — | — | — | — | |
| HippoRAGModel=GPT-4o-mini2026.04 | 33.59 | — | 48.17 | — | 28.59 | — | 54.84 | — | — | — | 47.92 | — | — | — | — | |
| CAMModel=GPT-4o-mini2026.04 | 33.55 | — | 44.14 | — | 18.23 | — | 50.58 | — | — | — | 44.1 | — | — | — | — | |
| LoCoMoModel=Qwen3-14B2026.02 | 33.37 | 24.26 | 31.49 | 16.42 | 13.92 | 11.02 | 25.46 | 24.82 | 49.17 | 35 | 32.42 | 25.02 | — | — | — | |
| Full KVBackbone=Qwen2.5-3B2025.09 | 33.2 | — | 22.9 | — | 12.3 | — | 49.1 | — | — | — | 38.4 | — | — | — | — | |
| A-MEMModel=GPT-4o2026.02 | 32.86 | 23.76 | 39.41 | 31.23 | 17.1 | 15.84 | 48.43 | 42.97 | 36.35 | 35.53 | 40.53 | 35.36 | — | — | — | |
| A-MEMModel=GPT-4o, Token Length=1,2162026.04 | 32.86 | 23.76 | 39.41 | 31.23 | 17.1 | 15.84 | 48.43 | 42.97 | 36.35 | 35.53 | — | — | — | — | — | |
| Original - Prompt CacheBackbone=Llama-3.1-8B-Instruct, Attention Mode=Prompt Cache2026.05 | 32.61 | — | 10.75 | — | 50.65 | — | 25.75 | — | 15.37 | — | 27.03 | — | — | — | — | |
| EPICACHEBackbone=Qwen2.5-7B, M (KV Cache Size)=6K2025.09 | 32.6 | — | 24.6 | — | 15.1 | — | 46.5 | — | — | — | 37.5 | — | — | — | — | |
| EPICACHEBackbone=Qwen2.5-7B, M (KV Cache Size)=8K2025.09 | 32.6 | — | 28.1 | — | 15.3 | — | 52.7 | — | — | — | 41.6 | — | — | — | — | |
| Full ContextModel=Qwen2.5-14B2026.04 | 32.59 | — | 27.36 | — | 13.5 | — | 55.35 | — | — | — | 42.74 | — | — | — | — | |
| NemoriLLM=GPT-4o-mini, Tokens=4,7672026.02 | 32.36 | — | 55.99 | — | 29.19 | — | 46.33 | — | — | — | 44.72 | — | — | — | — | |
| RAGModel=GPT-4o-mini2026.04 | 32.17 | — | 30.77 | — | 23.21 | — | 52.19 | — | — | — | 42.25 | — | — | — | — | |
| MEMFLYModel=GPT-4o-mini2026.02 | 32.11 | 24.48 | 46.61 | 31.84 | 23.98 | 16.84 | 44.74 | 38.17 | 51.48 | 51.96 | 43.76 | 37.27 | — | — | — | |
| LightMemLLM=GPT-4o-mini, Tokens=8152026.02 | 32.11 | — | 53.79 | — | 26.14 | — | 47.64 | — | — | — | 44.73 | — | — | — | — | |
| Mem0Model=Qwen2.5-14B2026.04 | 31.73 | — | 28.96 | — | 15.03 | — | 42.58 | — | — | — | 36.04 | — | — | — | — | |
| EPICACHEBackbone=Qwen2.5-3B, M (KV Cache Size)=8K2025.09 | 31.3 | — | 17 | — | 10 | — | 41.7 | — | — | — | 32.7 | — | — | — | — | |
| MEMFLYModel=Qwen3-14B2026.02 | 30.8 | 23.13 | 29.25 | 24.56 | 14.11 | 11.03 | 42.25 | 35.52 | 26.59 | 25.02 | 33.65 | 28.45 | — | — | — | |
| SnapKVBackbone=Qwen2.5-7B, M (KV Cache Size)=8K2025.09 | 30.8 | — | 13.2 | — | 14.9 | — | 39.4 | — | — | — | 30.8 | — | — | — | — | |
| MEMGPTModel=GPT-4o2026.02 | 30.36 | 22.83 | 17.29 | 13.18 | 12.24 | 11.87 | 60.16 | 53.35 | 34.96 | 34.25 | 41.02 | 36.23 | — | — | — | |
| MemGPTModel=GPT-4o, Token Length=16,9872026.04 | 30.36 | 22.83 | 17.29 | 13.18 | 12.24 | 11.87 | 60.16 | 53.35 | 34.96 | 34.25 | — | — | — | — | — | |
| Original - Prompt CacheBackbone=Qwen3-8B, Attention Mode=Prompt Cache2026.05 | 29.71 | — | 15.37 | — | 48.27 | — | 25.62 | — | 17.51 | — | 27.3 | — | — | — | — | |
| Original - Prompt CacheBackbone=Qwen3-14B, Attention Mode=Prompt Cache2026.05 | 29.68 | — | 7.34 | — | 51.36 | — | 28.87 | — | 22.97 | — | 28.04 | — | — | — | — | |
| EPICACHEBackbone=Qwen2.5-7B, M (KV Cache Size)=4K2025.09 | 29 | — | 20.9 | — | 14.1 | — | 38.5 | — | — | — | 31.6 | — | — | — | — | |
| LightMemModel=GPT-4o-mini, Token Length=1,1502026.04 | 28.85 | 21.4 | 46.2 | 37.1 | 13.5 | 12.9 | 45.8 | 38.2 | 54.5 | 52.1 | — | — | — | — | — | |
| KeyDiffBackbone=Qwen2.5-7B, M (KV Cache Size)=8K2025.09 | 28.3 | — | 23.7 | — | 15.9 | — | 36.4 | — | — | — | 30.8 | — | — | — | — | |
| MEMFLYModel=Qwen3-8B2026.02 | 28.24 | 22.76 | 38.39 | 33.64 | 15.43 | 13.81 | 42.09 | 36.57 | 43.79 | 43.14 | 38.62 | 34.51 | — | — | — | |
| RAGModel=Qwen2.5-14B2026.04 | 28.11 | — | 24.73 | — | 20.42 | — | 49.79 | — | — | — | 38.77 | — | — | — | — | |
| Locas-GLUBackbone=Qwen3-4B-Base, Context Setting=Full Attention, Adaptation Method=TTT2026.02 | 28.1 | — | 18.1 | — | 7.1 | — | 47.6 | — | — | — | — | — | -19.8 | — | — | |
| LoCoMoModel=GPT-4o2026.02 | 28 | 18.47 | 9.09 | 5.78 | 16.47 | 14.8 | 61.56 | 54.19 | 52.61 | 51.13 | 44.12 | 38.7 | — | — | — | |
| LoCoMoModel=GPT-4o, Token Length=16,9102026.04 | 28 | 18.47 | 9.09 | 5.78 | 16.47 | 14.8 | 61.56 | 54.19 | 52.61 | 51.13 | — | — | — | — | — | |
| SnapKVBackbone=Qwen2.5-7B, M (KV Cache Size)=6K2025.09 | 27.8 | — | 9.1 | — | 15.2 | — | 30.2 | — | — | — | 24.4 | — | — | — | — | |
| InfiniPotBackbone=Qwen2.5-7B, M (KV Cache Size)=8K2025.09 | 27.7 | — | 14 | — | 14 | — | 32.7 | — | — | — | 29.4 | — | — | — | — | |
| Original - SuperpositionBackbone=Qwen3-14B, Attention Mode=Superposition2026.05 | 27.62 | — | 4.39 | — | 42.59 | — | 20.63 | — | 19.41 | — | 22.93 | — | — | — | — | |
| HippoRAGModel=Qwen2.5-14B2026.04 | 27.57 | — | 30.66 | — | 19.74 | — | 42.45 | — | — | — | 35.85 | — | — | — | — | |
| EPICACHEBackbone=Qwen2.5-3B, M (KV Cache Size)=6K2025.09 | 27.3 | — | 13.3 | — | 10.8 | — | 37.2 | — | — | — | 28.8 | — | — | — | — | |
| EPICACHEBackbone=Qwen2.5-3B, M (KV Cache Size)=4K2025.09 | 27.1 | — | 10.7 | — | 11.4 | — | 29.7 | — | — | — | 24.1 | — | — | — | — | |
| A-MEMModel=GPT-4o-mini2026.02 | 27.02 | 20.09 | 45.85 | 36.67 | 12.14 | 12 | 44.65 | 37.06 | 50.03 | 49.47 | 41.97 | 36.16 | — | — | — | |
| A-MemLLM=GPT-4o-mini, Tokens=2,7122026.02 | 27.02 | — | 45.85 | — | 12.14 | — | 44.65 | — | — | — | 39.65 | — | — | — | — | |
| A-MEMModel=GPT-4o-mini, Token Length=2,5202026.04 | 27.02 | 20.09 | 45.85 | 36.67 | 12.14 | 12 | 44.65 | 37.06 | 50.03 | 49.47 | — | — | — | — | — | |
| A-MemModel=GPT-4o-mini2026.04 | 27.02 | — | 45.85 | — | 12.14 | — | 44.65 | — | — | — | 39.65 | — | — | — | — | |
| A-MEM (self-rep.)LLM Backbone=GPT-4o-mini2026.06 | 27.02 | — | 45.85 | — | 12.14 | — | 44.65 | — | — | — | — | — | — | — | — | |
| MEMGPTModel=GPT-4o-mini2026.02 | 26.65 | 17.72 | 25.52 | 19.44 | 9.15 | 7.44 | 41.04 | 34.34 | 43.29 | 42.73 | 35.45 | 30.16 | — | — | — | |
| MemGPTModel=GPT-4o-mini, Token Length=16,9772026.04 | 26.65 | 17.72 | 25.52 | 19.44 | 9.15 | 7.44 | 41.04 | 34.34 | 43.29 | 42.73 | — | — | — | — | — | |
| EPICACHEBackbone=Qwen2.5-7B, M (KV Cache Size)=2K2025.09 | 26.4 | — | 15.4 | — | 13.2 | — | 29.3 | — | — | — | 24.9 | — | — | — | — | |
| KeyDiffBackbone=Qwen2.5-7B, M (KV Cache Size)=6K2025.09 | 26.3 | — | 25.2 | — | 14.3 | — | 29.8 | — | — | — | 27.2 | — | — | — | — | |
| MEMORYBANKModel=Qwen3-14B2026.02 | 25.97 | 18.16 | 25.37 | 18.76 | 13.52 | 11.69 | 34.92 | 30.6 | 21.94 | 17.56 | 28.16 | 23.08 | — | — | — | |
| KVzipBackbone=Qwen2.5-7B, M (KV Cache Size)=8K2025.09 | 25.6 | — | 23.8 | — | 13 | — | 34.8 | — | — | — | 29.5 | — | — | — | — | |
| Original - SuperpositionBackbone=Llama-3.1-8B-Instruct, Attention Mode=Superposition2026.05 | 25.51 | — | 15.51 | — | 40.48 | — | 28.64 | — | 17.42 | — | 25.51 | — | — | — | — | |
| Locas-GLUBackbone=Qwen3-1.7B-Base, Context Setting=Full Attention, Adaptation Method=TTT2026.02 | 25.2 | — | 34.1 | — | 14.1 | — | 41.6 | — | — | — | — | — | -28.7 | — | — | |
| LoCoMoModel=Qwen3-8B2026.02 | 25.09 | 15.73 | 32.82 | 27.14 | 14.47 | 13.35 | 20.18 | 18.39 | 46.77 | 40.81 | 28.62 | 24.22 | — | — | — | |
| LoCoMoModel=GPT-4o-mini2026.02 | 25.02 | 19.75 | 18.41 | 14.77 | 12.04 | 11.16 | 40.36 | 29.05 | 69.23 | 68.75 | 39.74 | 33.47 | — | — | — | |
| LoCoMoModel=GPT-4o-mini, Token Length=16,9102026.04 | 25.02 | 19.75 | 18.41 | 14.77 | 12.04 | 11.16 | 40.36 | 29.05 | 69.23 | 68.75 | — | — | — | — | — | |
| No memoryLLM Backbone=GPT-4o-mini2026.06 | 25.02 | — | 18.41 | — | 12.04 | — | 40.36 | — | — | — | — | — | — | — | — | |
| Full ContextModel=GPT-4o-mini2026.04 | 25.01 | — | 23.83 | — | 15.49 | — | 45.89 | — | — | — | 35.57 | — | — | — | — | |
| A-MEMModel=Qwen3-8B2026.02 | 24.3 | 16.9 | 34.5 | 23.1 | 13.1 | 12.2 | 38.1 | 33.3 | 31 | 30.1 | 32.76 | 27.58 | — | — | — | |
| TempLoRABackbone=Qwen3-4B-Base, Context Setting=Full Attention, Adaptation Method=TTT2026.02 | 24.3 | — | 17.2 | — | 7.9 | — | 41.5 | — | — | — | — | — | -24.3 | — | — | |
| KVzipBackbone=Qwen2.5-7B, M (KV Cache Size)=6K2025.09 | 24.3 | — | 19.5 | — | 12.5 | — | 27.2 | — | — | — | 24.2 | — | — | — | — | |
| MEMGPTModel=Qwen3-14B2026.02 | 24.12 | 15.41 | 25.48 | 19.04 | 13.44 | 12.64 | 34.74 | 32.41 | 27.11 | 24.32 | 28.99 | 25.06 | — | — | — | |
| InfiniPotBackbone=Qwen2.5-3B, M (KV Cache Size)=8K2025.09 | 23.9 | — | 18.8 | — | 13.2 | — | 25.1 | — | — | — | 22.8 | — | — | — | — | |
| Full AttentionBackbone=Qwen3-1.7B-Base, Context Setting=Full Attention, Adaptation Method=None2026.02 | 23.8 | — | 33.5 | — | 14 | — | 37.3 | — | — | — | — | — | -33 | — | — | |
| SnapKVBackbone=Qwen2.5-7B, M (KV Cache Size)=4K2025.09 | 23.8 | — | 8 | — | 13.1 | — | 24.2 | — | — | — | 20.1 | — | — | — | — | |
| EPICACHEBackbone=Qwen2.5-3B, M (KV Cache Size)=2K2025.09 | 23.6 | — | 7.6 | — | 13.4 | — | 23.8 | — | — | — | 19.8 | — | — | — | — |