Multi-Hop Reasoning on LoCoMo
41.62F1 ScoreR2-Mem
Evaluation results
| Method | Links | ||
|---|---|---|---|
| R2-MemModel=Qwen2.5-7B2026.05 | 41.62 | 32.34 | |
| GAMModel=Qwen2.5-7B2026.05 | 38.09 | 29.02 | |
| R2-MemModel=Llama3.1-8B2026.05 | 37.75 | 29.28 | |
| SCG-Mem (Ours)Model=Llama 3.2 1B2026.04 | 36.66 | 29.06 | |
| Memory R1Model=Qwen2.5-7B2026.05 | 36.55 | 28.71 | |
| GAMModel=Llama3.1-8B2026.05 | 36.23 | 27.71 | |
| GAMModel=GPT-4o-mini2026.04 | 35.88 | 27.96 | |
| Memory R1Model=Llama3.1-8B2026.05 | 35.36 | 26.43 | |
| GAMModel=Qwen 2.5-7B2026.04 | 35.32 | 25.77 | |
| MemoryOSModel=GPT-4o-mini2026.04 | 35.27 | 25.22 | |
| GAMModel=Qwen 2.5-14B2026.04 | 33.32 | 25.38 | |
| Mem0Model=Qwen 2.5-7B2026.04 | 32.86 | 23.99 | |
| Mem0Model=GPT-4o-mini2026.04 | 32.78 | 20.87 | |
| Mem0Model=Qwen 2.5-14B2026.04 | 31.45 | 24.59 | |
| MemoryOSModel=Qwen 2.5-7B2026.04 | 29.37 | 21.4 | |
| GAMModel=Llama 3.2-3B2026.04 | 28.74 | 22.35 | |
| SCG-Mem (Ours)Model=Qwen2.5 3B2026.04 | 28.49 | 21.21 | |
| MemoryOSModel=Llama3.1-8B2026.05 | 27.55 | 18.91 | |
| Mem0Model=Llama 3.2-3B2026.04 | 27.51 | 18.51 | |
| MemoryOSModel=Qwen 2.5-14B2026.04 | 27.46 | 19.14 | |
| A-MemModel=GPT-4o-mini2026.04 | 27.02 | 20.09 | |
| MemoryOSModel=Qwen2.5-7B2026.05 | 26.72 | 17.9 | |
| MemgptModel=GPT-4o-mini2026.04 | 26.65 | 17.72 | |
| A-MemModel=Qwen 2.5-14B2026.04 | 26.55 | 16.56 | |
| LIGHTMEMModel=Llama3.1-8B2026.05 | 24.42 | 18.66 | |
| A-MEMModel=Llama3.1-8B2026.05 | 23.69 | 15.28 | |
| LIGHTMEMModel=Qwen2.5-7B2026.05 | 23.45 | 17.61 | |
| A-MEMModel=Qwen2.5-7B2026.05 | 22.59 | 16.28 | |
| A-MemModel=Qwen 2.5-7B2026.04 | 21.85 | 13.4 | |
| MemoryOSModel=Llama 3.2-3B2026.04 | 21.03 | 13.75 | |
| Mem0Model=Llama3.1-8B2026.05 | 20.43 | 15.76 | |
| SCG-Mem (Ours)Model=Llama 3.2 3B2026.04 | 20.16 | 12.67 | |
| SCG-Mem (Ours)Model=Qwen2.5 1.5B2026.04 | 20.06 | 16.22 | |
| Mem0Model=Qwen2.5-7B2026.05 | 19.23 | 14.17 | |
| A-MEMModel=Llama 3.2 1B2026.04 | 19.06 | 11.71 | |
| A-MEMModel=Qwen2.5 1.5B2026.04 | 18.23 | 11.94 | |
| A-MemModel=Llama 3.2-3B2026.04 | 17.44 | 11.74 | |
| A-MEMModel=Llama 3.2 3B2026.04 | 17.44 | 11.74 | |
| RAGModel=Llama3.1-8B2026.05 | 13.29 | 10.66 | |
| MemoryBankModel=Llama 3.2 1B2026.04 | 13.18 | 10.03 | |
| RAGModel=Qwen2.5-7B2026.05 | 12.84 | 11.02 | |
| A-MEMModel=Qwen2.5 3B2026.04 | 12.57 | 9.01 | |
| LoCoMoModel=Llama 3.2 1B2026.04 | 11.25 | 9.18 | |
| MemoryBankModel=Qwen2.5 1.5B2026.04 | 11.14 | 8.25 | |
| MemGPTModel=Qwen2.5 1.5B2026.04 | 10.44 | 7.61 | |
| MemGPTModel=Llama 3.2 1B2026.04 | 9.19 | 6.96 | |
| ReadAgentModel=GPT-4o-mini2026.04 | 9.15 | 6.48 | |
| LoCoMoModel=Qwen2.5 1.5B2026.04 | 9.05 | 6.55 | |
| MemgptModel=Qwen 2.5-14B2026.04 | 8.37 | 5.42 | |
| MemgptModel=Llama 3.2-3B2026.04 | 7.59 | 5.09 | |
| MemorybankModel=Llama 3.2-3B2026.04 | 7.35 | 5.34 | |
| LoCoMoModel=Llama 3.2 3B2026.04 | 6.88 | 5.77 | |
| ReadAgentModel=Qwen2.5 1.5B2026.04 | 6.61 | 4.93 | |
| MemorybankModel=Qwen 2.5-7B2026.04 | 6.22 | 5.39 | |
| MemoryBankModel=Llama 3.2 3B2026.04 | 6.19 | 4.47 | |
| MemgptModel=Qwen 2.5-7B2026.04 | 6.17 | 4.97 | |
| ReadAgentModel=Llama 3.2 1B2026.04 | 5.96 | 5.12 | |
| ReadAgentModel=Qwen 2.5-7B2026.04 | 5.37 | 4.02 | |
| MemGPTModel=Llama 3.2 3B2026.04 | 5.32 | 3.99 | |
| MemGPTModel=Qwen2.5 3B2026.04 | 5.07 | 4.31 | |
| MemorybankModel=GPT-4o-mini2026.04 | 5 | 4.77 | |
| LoCoMoModel=Qwen2.5 3B2026.04 | 4.61 | 4.29 | |
| ReadAgentModel=Qwen 2.5-14B2026.04 | 4.33 | 2.99 | |
| MemorybankModel=Qwen 2.5-14B2026.04 | 3.65 | 3.16 | |
| MemoryBankModel=Qwen2.5 3B2026.04 | 3.6 | 3.39 | |
| ReadAgentModel=Qwen2.5 3B2026.04 | 2.47 | 1.78 | |
| ReadAgentModel=Llama 3.2 3B2026.04 | 2.47 | 1.78 | |
| ReadAgentModel=Llama 3.2-3B2026.04 | 2.05 | 1.96 |