Multi-hop Question Answering on LoCoMo
48.35F1Membox
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| MemboxModel=GPT-4o2026.01 | 48.35 | 35.1 | — | — | — | — | — | — | |
| LightMEM+AMAModel=GPT-4o2026.01 | 46.35 | — | — | 33.82 | — | — | — | — | |
| ShardMemoModel=GPT-OSS-120B2026.01 | 46.28 | — | — | 39.65 | — | — | — | — | |
| Full Context2026.02 | 45.9 | 35.6 | 76.6 | — | — | — | — | — | |
| LightMEMModel=GPT-4o2026.01 | 44.86 | — | — | 32.44 | — | — | — | — | |
| FullContextLLM=GPT-4.1-mini2026.01 | 44.2 | — | 77.2 | 33.7 | — | — | — | — | |
| MEMIRBackbone=GPT-4.12026.05 | 42.9 | — | — | 35 | 71.7 | — | — | — | |
| MEMORARetriever Type=Policy Retriever2026.02 | 42.8 | 33.7 | 78.7 | — | — | — | — | — | |
| NemoriLLM=GPT-4.1-mini2026.01 | 42.8 | — | 76.2 | 33 | — | — | — | — | |
| O-MemBackbone=GPT-4.12026.05 | 42.64 | — | — | 34.08 | — | — | — | — | |
| Mem0Model=GPT-4o, Local re-implementation=true, k=302026.01 | 42.57 | 30.92 | — | — | — | — | — | — | |
| MEMIRBackbone=GPT-4.1-mini2026.05 | 42.5 | — | — | 34.1 | 70.2 | — | — | — | |
| SimpleMemBackbone=GPT-4.12026.05 | 42.3 | — | — | 34.4 | 75.2 | — | — | — | |
| Nemori*2026.02 | 41.7 | 31.9 | 75.1 | — | — | — | — | — | |
| MEMORARetriever Type=Semantic Retriever2026.02 | 41.7 | 32.1 | 78.4 | — | — | — | — | — | |
| LangMem*2026.02 | 41.5 | 32.5 | 71 | — | — | — | — | — | |
| LangMemLLM=GPT-4.1-mini2026.01 | 41.5 | — | 71 | 32.5 | — | — | — | — | |
| LangMemBackbone=GPT-4.1-mini2026.05 | 41.5 | — | — | 32.5 | 71 | — | — | — | |
| Mem0Backbone=GPT-4.12026.05 | 41.3 | — | — | 32.5 | 70.3 | — | — | — | |
| GAMModel=GPT-OSS-120B2026.01 | 41.17 | — | — | 33.06 | — | — | — | — | |
| LangMemBackbone=GPT-4.12026.05 | 41.11 | — | — | 32.09 | — | — | — | — | |
| Nemori+AMAModel=GPT-4o-mini2026.01 | 40.97 | — | — | 29.36 | — | — | — | — | |
| NEMORIBackbone=GPT-4.1-mini2026.05 | 40.8 | — | — | 31.7 | 74.8 | — | — | — | |
| MemoryOSModel=GPT-4o, Token Length=2,0002026.04 | 40.23 | — | — | 31.89 | — | — | — | — | |
| HeLa-MemModel=GPT-4o-mini, Token Length=1,0102026.04 | 40.14 | — | — | 31.26 | — | — | — | — | |
| Mem0LLM=GPT-4.1-mini2026.01 | 40.1 | — | 68.2 | 30.3 | — | — | — | — | |
| Mem0Backbone=GPT-4.1-mini2026.05 | 40.1 | — | — | 30.3 | 68.2 | — | — | — | |
| NEMORIBackbone=GPT-4.12026.05 | 40.1 | — | — | 32.6 | 74 | — | — | — | |
| MemboxModel=GPT-4o-mini2026.01 | 39.88 | 26.39 | — | — | — | — | — | — | |
| NemoriModel=GPT-4o2026.01 | 39.75 | — | — | 28.92 | — | — | — | — | |
| SimpleMemBackbone=GPT-4.1-mini2026.05 | 39.6 | — | — | 31.5 | 73.8 | — | — | — | |
| Nemori+AMAModel=GPT-4o2026.01 | 39.52 | — | — | 28.22 | — | — | — | — | |
| HeLa-MemModel=GPT-4o, Token Length=1,0362026.04 | 39.12 | — | — | 29.82 | — | — | — | — | |
| Mem0Model=GPT-4o-mini2026.01 | 38.72 | 27.13 | — | — | — | — | — | — | |
| MEM0Model=GPT-OSS-120B2026.01 | 38.6 | — | — | 26.54 | — | — | — | — | |
| Memory-R2Training regime=Trained, Base Model=Qwen2.5-7B-Instruct, Answer Agent=Qwen2.5-7B-Instruct2026.05 | 38.41 | — | 80.93 | 30.9 | — | — | — | — | |
| MemoryOSModel=GPT-4o-mini, Token Length=2,0002026.04 | 38.39 | — | — | 29.52 | — | — | — | — | |
| MemoryOSModel=Qwen2.5-14b2026.04 | 38.19 | — | — | 29.26 | — | — | — | — | |
| LightMEMModel=GPT-4o-mini2026.01 | 37.41 | — | — | 25.9 | — | — | — | — | |
| LightMEM+AMAModel=GPT-4o-mini2026.01 | 37.17 | — | — | 25.74 | — | — | — | — | |
| NemoriModel=GPT-4o-mini2026.01 | 37.06 | — | — | 24.96 | — | — | — | — | |
| Mem0Model=GPT-4o-mini, Local re-implementation=true, k=302026.01 | 36.83 | 26.5 | — | — | — | — | — | — | |
| HeLa-MemModel=Qwen2.5-14b, Token Length=9442026.04 | 36.59 | — | — | 27.02 | — | — | — | — | |
| Memory-R2(OSS)Training regime=Trained, Base Model=Qwen2.5-7B-Instruct, Answer Agent=GPT-OSS-120B2026.05 | 36.37 | — | 85.07 | 29.79 | — | — | — | — | |
| MemAgentTraining regime=Trained, Base Model=Qwen2.5-7B-Instruct, Answer Agent=Qwen2.5-7B-Instruct2026.05 | 35.95 | — | 70.65 | 24.56 | — | — | — | — | |
| MemoryOSBackbone=GPT-4.12026.05 | 35.9 | — | — | 26 | 66.4 | — | — | — | |
| MEMORYOSModel=GPT-OSS-120B2026.01 | 34.13 | — | — | 26.02 | — | — | — | — | |
| MemoryOSBackbone=GPT-4.1-mini2026.05 | 34 | — | — | 25.8 | 62.4 | — | — | — | |
| HIMEMBackbone=GPT-4.12026.05 | 33.8 | — | — | 25.7 | 69.4 | — | — | — | |
| Memory-R1Training regime=Trained, Base Model=Qwen2.5-7B-Instruct, Answer Agent=Qwen2.5-7B-Instruct2026.05 | 33.64 | — | 62.34 | 26.06 | — | — | — | — | |
| A-MEMModel=GPT-4o2026.01 | 32.97 | — | — | 21.93 | — | — | — | — | |
| A-MEMModel=GPT-4o2026.01 | 32.86 | 23.76 | — | — | — | — | — | — | |
| A-MemModel=GPT-4o, Token Length=1,2162026.04 | 32.86 | — | — | 23.76 | — | — | — | — | |
| Mem02026.02 | 32.6 | 23.6 | 62.4 | — | — | — | — | — | |
| RAG2026.02 | 32.4 | 22.2 | 55.7 | — | — | — | — | — | |
| SwiftMemLLM=GPT-4.1-mini2026.01 | 32 | — | 58.9 | 37.1 | — | — | — | — | |
| SwiftMemBackbone=GPT-4.1-mini2026.05 | 32 | — | — | 37.1 | 58.9 | — | — | — | |
| HIPPOCAMPUS2026.02 | 31.97 | — | — | 31.85 | — | 3.22 | — | — | |
| MEM0Model=Qwen2.5-14b2026.04 | 31.73 | — | — | 24.82 | — | — | — | — | |
| A-MEMModel=GPT-4o, Local re-implementation=true, k=302026.01 | 31.66 | 23.34 | — | — | — | — | — | — | |
| Zep*2026.02 | 30.5 | 20.4 | 53.7 | — | — | — | — | — | |
| ZepLLM=GPT-4.1-mini2026.01 | 30.5 | — | 53.7 | 20.4 | — | — | — | — | |
| ZepBackbone=GPT-4.1-mini2026.05 | 30.5 | — | — | 20.4 | 53.7 | — | — | — | |
| A-MemBackbone=GPT-4.1-mini2026.05 | 30.4 | — | — | 20 | 55.7 | — | — | — | |
| MemGPTModel=GPT-4o2026.01 | 30.36 | — | — | 22.83 | — | — | — | — | |
| MEMGPTModel=GPT-4o2026.01 | 30.36 | 22.83 | — | — | — | — | — | — | |
| MemGPTModel=GPT-4o, Token Length=16,9872026.04 | 30.36 | — | — | 22.83 | — | — | — | — | |
| HIMEMBackbone=GPT-4.1-mini2026.05 | 30.3 | — | — | 23.7 | 71.5 | — | — | — | |
| LIGHTMEMModel=GPT-OSS-120B2026.01 | 30.28 | — | — | 25.49 | — | — | — | — | |
| MemOS2026.02 | 30.11 | — | — | 30.91 | — | 2.56 | — | — | |
| MemoryOSTraining regime=Training-free, Base Model=Qwen2.5-7B-Instruct, Answer Agent=Qwen2.5-7B-Instruct2026.05 | 29.55 | — | 48.12 | 22.59 | — | — | — | — | |
| LightMemBackbone=GPT-4.12026.05 | 28.7 | — | — | 25.6 | 68.3 | — | — | — | |
| MemoryOS2026.02 | 28.61 | — | — | 26.81 | — | 1.79 | — | — | |
| RAGModel=GPT-OSS-120B2026.01 | 28.33 | — | — | 21.63 | — | — | — | — | |
| A-MEM+AMAModel=GPT-4o2026.01 | 28.03 | — | — | 19.95 | — | — | — | — | |
| LoCoMoModel=GPT-4o2026.01 | 28 | — | — | 18.47 | — | — | — | — | |
| LoCoMoModel=GPT-4o2026.01 | 28 | 18.47 | — | — | — | — | — | — | |
| LoCoMoModel=GPT-4o, Token Length=16,9102026.04 | 28 | — | — | 18.47 | — | — | — | — | |
| Vanilla LLMModel=GPT-OSS-120B2026.01 | 27.1 | — | — | 21.45 | — | — | — | — | |
| A-MEMModel=GPT-4o-mini, Local re-implementation=true, k=302026.01 | 27.08 | 20.46 | — | — | — | — | — | — | |
| A-MEMModel=GPT-4o-mini2026.01 | 27.02 | 20.09 | — | — | — | — | — | — | |
| A-MemModel=GPT-4o-mini, Token Length=2,5202026.04 | 27.02 | — | — | 20.09 | — | — | — | — | |
| A-MEMModel=GPT-OSS-120B2026.01 | 26.81 | — | — | 20.45 | — | — | — | — | |
| A-MemBackbone=GPT-4.12026.05 | 26.7 | — | — | 18.4 | 56.3 | — | — | — | |
| MemGPTModel=GPT-4o-mini2026.01 | 26.65 | — | — | 17.72 | — | — | — | — | |
| MEMGPTModel=GPT-4o-mini2026.01 | 26.65 | 17.72 | — | — | — | — | — | — | |
| MemGPTModel=GPT-4o-mini, Token Length=16,9772026.04 | 26.65 | — | — | 17.72 | — | — | — | — | |
| LightMemModel=Qwen2.5-14b2026.04 | 25.45 | — | — | 19.61 | — | — | — | — | |
| LoCoMoModel=GPT-4o-mini2026.01 | 25.02 | — | — | 19.75 | — | — | — | — | |
| LoCoMoModel=GPT-4o-mini2026.01 | 25.02 | 19.75 | — | — | — | — | — | — | |
| LoCoMoModel=GPT-4o-mini, Token Length=16,9102026.04 | 25.02 | — | — | 19.75 | — | — | — | — | |
| LoCoMoBackbone=GPT-4.1-mini2026.05 | 25 | — | — | 21.6 | — | — | — | — | |
| Mem0Training regime=Training-free, Base Model=Qwen2.5-7B-Instruct, Answer Agent=Qwen2.5-7B-Instruct2026.05 | 24.96 | — | 61.92 | 18.05 | — | — | — | — | |
| LightMemBackbone=GPT-4.1-mini2026.05 | 24.9 | — | — | 21.7 | 69.6 | — | — | — | |
| A-MemModel=Qwen2.5-14b, Token Length=1,3002026.04 | 22.09 | — | — | 15.28 | — | — | — | — | |
| A-MEM+AMAModel=GPT-4o-mini2026.01 | 21.5 | — | — | 15.2 | — | — | — | — | |
| HeLa-MemModel=Qwen2.5-3b, Token Length=1,0722026.04 | 20.12 | — | — | 14.59 | — | — | — | — | |
| A-MEMModel=GPT-4o-mini2026.01 | 20.1 | — | — | 15 | — | — | — | — | |
| RAGLLM=GPT-4.1-mini2026.01 | 20.1 | — | 31.7 | 12.8 | — | — | — | — | |
| MemoryOSModel=Qwen2.5-3b, Token Length=2,0002026.04 | 19.2 | — | — | 14.84 | — | — | — | — |