Dialogue-Style Memory Reasoning on LoCoMo (Metrics: EM, LJ)
59.98EMREAL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| REALModel=DeepSeek-V3, Memory Category=Graph-Based Memory Baselines2026.06 | 59.98 | 73.76 | |
| REALModel=LLaMA-3.3-70B-Instruct, Memory Category=Graph-Based Memory Baselines2026.06 | 58.69 | 72.83 | |
| A-MEMModel=LLaMA-3.3-70B-Instruct, Memory Category=Graph-Based Memory Baselines2026.06 | 55.38 | 70.72 | |
| A-MEMModel=DeepSeek-V3, Memory Category=Graph-Based Memory Baselines2026.06 | 55.12 | 68.8 | |
| REALModel=Qwen3-32B, Memory Category=Graph-Based Memory Baselines2026.06 | 54.88 | 69.47 | |
| Mem0gModel=DeepSeek-V3, Memory Category=Graph-Based Memory Baselines2026.06 | 53.96 | 67.12 | |
| Mem0Model=DeepSeek-V3, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 52.65 | 66.67 | |
| Mem0gModel=LLaMA-3.3-70B-Instruct, Memory Category=Graph-Based Memory Baselines2026.06 | 52.12 | 66.35 | |
| Mem0gModel=Qwen3-32B, Memory Category=Graph-Based Memory Baselines2026.06 | 51.74 | 66.3 | |
| Mem0Model=LLaMA-3.3-70B-Instruct, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 50.67 | 64.85 | |
| A-MEMModel=Qwen3-32B, Memory Category=Graph-Based Memory Baselines2026.06 | 50.65 | 64.51 | |
| Vanilla Memory GraphModel=DeepSeek-V3, Memory Category=Graph-Based Memory Baselines2026.06 | 49.43 | 62.64 | |
| Mem0Model=Qwen3-32B, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 49.11 | 63.05 | |
| MemGPTModel=DeepSeek-V3, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 48.32 | 61.28 | |
| Vanilla Memory GraphModel=LLaMA-3.3-70B-Instruct, Memory Category=Graph-Based Memory Baselines2026.06 | 47.4 | 61.02 | |
| MemGPTModel=LLaMA-3.3-70B-Instruct, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 46.16 | 59.4 | |
| Vanilla Memory GraphModel=Qwen3-32B, Memory Category=Graph-Based Memory Baselines2026.06 | 45.92 | 59.28 | |
| Flat Memory ChunkModel=DeepSeek-V3, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 45.6 | 58.53 | |
| MemGPTModel=Qwen3-32B, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 44.74 | 57.83 | |
| Flat Memory ChunkModel=LLaMA-3.3-70B-Instruct, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 43.87 | 56.48 | |
| Flat Memory ChunkModel=Qwen3-32B, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 42.36 | 55.1 | |
| Full ContextModel=DeepSeek-V3, Memory Category=Full Context2026.06 | 35.24 | 46.84 | |
| Full ContextModel=LLaMA-3.3-70B-Instruct, Memory Category=Full Context2026.06 | 34.8 | 45.66 | |
| Full ContextModel=Qwen3-32B, Memory Category=Full Context2026.06 | 31.06 | 43.28 |