Long-Form Document-Style Memory Reasoning on 2WikiMultihopQA (EM, LJ)
45.65Exact Match (EM)REAL
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| REALModel=LLaMA-3.3-70B-Instruct, Memory Category=Graph-Based Memory Baselines2026.06 | 45.65 | 48.7 | |
| REALModel=DeepSeek-V3, Memory Category=Graph-Based Memory Baselines2026.06 | 43.62 | 46.17 | |
| Mem0gModel=LLaMA-3.3-70B-Instruct, Memory Category=Graph-Based Memory Baselines2026.06 | 41.58 | 45.79 | |
| Vanilla Memory GraphModel=DeepSeek-V3, Memory Category=Graph-Based Memory Baselines2026.06 | 40.58 | 43.87 | |
| A-MEMModel=DeepSeek-V3, Memory Category=Graph-Based Memory Baselines2026.06 | 39.95 | 42.33 | |
| REALModel=Qwen3-32B, Memory Category=Graph-Based Memory Baselines2026.06 | 39.07 | 42.79 | |
| A-MEMModel=LLaMA-3.3-70B-Instruct, Memory Category=Graph-Based Memory Baselines2026.06 | 38.75 | 41.65 | |
| Mem0gModel=DeepSeek-V3, Memory Category=Graph-Based Memory Baselines2026.06 | 38.04 | 40.06 | |
| Mem0Model=DeepSeek-V3, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 37.75 | 39.68 | |
| A-MEMModel=Qwen3-32B, Memory Category=Graph-Based Memory Baselines2026.06 | 36.52 | 39.16 | |
| Vanilla Memory GraphModel=Qwen3-32B, Memory Category=Graph-Based Memory Baselines2026.06 | 35.95 | 38.83 | |
| Mem0Model=LLaMA-3.3-70B-Instruct, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 35.55 | 38.16 | |
| Flat Memory ChunkModel=DeepSeek-V3, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 35.52 | 38.64 | |
| Mem0gModel=Qwen3-32B, Memory Category=Graph-Based Memory Baselines2026.06 | 35.05 | 37.8 | |
| MemGPTModel=DeepSeek-V3, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 34.46 | 36.65 | |
| Mem0Model=Qwen3-32B, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 33.55 | 37.65 | |
| Vanilla Memory GraphModel=LLaMA-3.3-70B-Instruct, Memory Category=Graph-Based Memory Baselines2026.06 | 33.52 | 35.67 | |
| Full ContextModel=DeepSeek-V3, Memory Category=Full Context2026.06 | 33.14 | 36.25 | |
| MemGPTModel=Qwen3-32B, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 32.57 | 36.86 | |
| MemGPTModel=LLaMA-3.3-70B-Instruct, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 32.15 | 34.62 | |
| Full ContextModel=LLaMA-3.3-70B-Instruct, Memory Category=Full Context2026.06 | 31.51 | 34.05 | |
| Flat Memory ChunkModel=LLaMA-3.3-70B-Instruct, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 30.56 | 32.63 | |
| Flat Memory ChunkModel=Qwen3-32B, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 30.49 | 33.67 | |
| Full ContextModel=Qwen3-32B, Memory Category=Full Context2026.06 | 29.93 | 32.45 |