Long-Form Document-Style Memory Reasoning on MuSiQue
30.34Exact Match (EM)REAL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| REALModel=DeepSeek-V3, Memory Category=Graph-Based Memory Baselines2026.06 | 30.34 | 41.73 | |
| REALModel=LLaMA-3.3-70B-Instruct, Memory Category=Graph-Based Memory Baselines2026.06 | 28.59 | 39.18 | |
| A-MEMModel=DeepSeek-V3, Memory Category=Graph-Based Memory Baselines2026.06 | 26.52 | 35.9 | |
| REALModel=Qwen3-32B, Memory Category=Graph-Based Memory Baselines2026.06 | 25.57 | 36.04 | |
| Vanilla Memory GraphModel=LLaMA-3.3-70B-Instruct, Memory Category=Graph-Based Memory Baselines2026.06 | 24.96 | 35.62 | |
| Mem0gModel=DeepSeek-V3, Memory Category=Graph-Based Memory Baselines2026.06 | 24.58 | 34.69 | |
| A-MEMModel=LLaMA-3.3-70B-Instruct, Memory Category=Graph-Based Memory Baselines2026.06 | 24.55 | 33.86 | |
| Mem0Model=DeepSeek-V3, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 23.57 | 34.61 | |
| Mem0gModel=LLaMA-3.3-70B-Instruct, Memory Category=Graph-Based Memory Baselines2026.06 | 23.25 | 32.98 | |
| A-MEMModel=Qwen3-32B, Memory Category=Graph-Based Memory Baselines2026.06 | 22.85 | 33.66 | |
| Vanilla Memory GraphModel=DeepSeek-V3, Memory Category=Graph-Based Memory Baselines2026.06 | 22.25 | 33.65 | |
| Mem0Model=LLaMA-3.3-70B-Instruct, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 21.75 | 32.65 | |
| Mem0gModel=Qwen3-32B, Memory Category=Graph-Based Memory Baselines2026.06 | 21.54 | 32.09 | |
| Vanilla Memory GraphModel=Qwen3-32B, Memory Category=Graph-Based Memory Baselines2026.06 | 20.84 | 31.76 | |
| MemGPTModel=DeepSeek-V3, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 20.59 | 32.36 | |
| Mem0Model=Qwen3-32B, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 19.51 | 30.62 | |
| Flat Memory ChunkModel=LLaMA-3.3-70B-Instruct, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 19.46 | 30.95 | |
| Full ContextModel=DeepSeek-V3, Memory Category=Full Context2026.06 | 19.37 | 30.25 | |
| MemGPTModel=LLaMA-3.3-70B-Instruct, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 18.46 | 29.61 | |
| Flat Memory ChunkModel=DeepSeek-V3, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 18.45 | 29.67 | |
| Flat Memory ChunkModel=Qwen3-32B, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 18.25 | 27.45 | |
| MemGPTModel=Qwen3-32B, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 17.44 | 25.96 | |
| Full ContextModel=LLaMA-3.3-70B-Instruct, Memory Category=Full Context2026.06 | 17.39 | 28.15 | |
| Full ContextModel=Qwen3-32B, Memory Category=Full Context2026.06 | 12.49 | 24.54 |