Long-Form Document-Style Memory Reasoning on HotpotQA
36.4EMREAL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| REALModel=DeepSeek-V3, Memory Category=Graph-Based Memory Baselines2026.06 | 36.4 | 59.06 | |
| REALModel=LLaMA-3.3-70B-Instruct, Memory Category=Graph-Based Memory Baselines2026.06 | 33.64 | 56.77 | |
| A-MEMModel=DeepSeek-V3, Memory Category=Graph-Based Memory Baselines2026.06 | 32.8 | 56.07 | |
| Mem0gModel=DeepSeek-V3, Memory Category=Graph-Based Memory Baselines2026.06 | 31.76 | 55.75 | |
| A-MEMModel=LLaMA-3.3-70B-Instruct, Memory Category=Graph-Based Memory Baselines2026.06 | 31.16 | 53.7 | |
| Mem0gModel=LLaMA-3.3-70B-Instruct, Memory Category=Graph-Based Memory Baselines2026.06 | 30.62 | 52.27 | |
| REALModel=Qwen3-32B, Memory Category=Graph-Based Memory Baselines2026.06 | 30.56 | 53.71 | |
| Vanilla Memory GraphModel=DeepSeek-V3, Memory Category=Graph-Based Memory Baselines2026.06 | 29.28 | 54.05 | |
| Mem0gModel=Qwen3-32B, Memory Category=Graph-Based Memory Baselines2026.06 | 29.03 | 51.37 | |
| Mem0Model=LLaMA-3.3-70B-Instruct, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 28.86 | 50.73 | |
| A-MEMModel=Qwen3-32B, Memory Category=Graph-Based Memory Baselines2026.06 | 28.25 | 50.63 | |
| MemGPTModel=DeepSeek-V3, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 28.15 | 52.37 | |
| Full ContextModel=DeepSeek-V3, Memory Category=Full Context2026.06 | 27.8 | 51.65 | |
| Mem0Model=DeepSeek-V3, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 27.65 | 51.79 | |
| Vanilla Memory GraphModel=LLaMA-3.3-70B-Instruct, Memory Category=Graph-Based Memory Baselines2026.06 | 27.65 | 50.69 | |
| MemGPTModel=LLaMA-3.3-70B-Instruct, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 27.57 | 49.67 | |
| Mem0Model=Qwen3-32B, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 26.85 | 48.97 | |
| Flat Memory ChunkModel=DeepSeek-V3, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 26.43 | 50.68 | |
| MemGPTModel=Qwen3-32B, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 25.54 | 48.66 | |
| Flat Memory ChunkModel=LLaMA-3.3-70B-Instruct, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 25.54 | 47.65 | |
| Vanilla Memory GraphModel=Qwen3-32B, Memory Category=Graph-Based Memory Baselines2026.06 | 25.51 | 48.04 | |
| Full ContextModel=LLaMA-3.3-70B-Instruct, Memory Category=Full Context2026.06 | 25.42 | 47.35 | |
| Flat Memory ChunkModel=Qwen3-32B, Memory Category=Flat-Text-Based Memory Baselines2026.06 | 25.25 | 47.1 | |
| Full ContextModel=Qwen3-32B, Memory Category=Full Context2026.06 | 24.35 | 45.8 |