Long-term conversational memory on LongMemEval Small
51.2F1 Score (%)RAPTOR
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| RAPTORLLM Backbone=Gemma-4-26B, Memory Category=Tree-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 51.2 | 65.67 | 14.56 | 30.8 | — | |
| MemTreeLLM Backbone=Gemma-4-26B, Memory Category=Tree-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 50.5 | 66.33 | 10.33 | 35 | — | |
| RAG (Dense)LLM Backbone=GPT-5.4-mini, Memory Category=No Structure, Augmentation Strategy=with GOAL-MEM2026.05 | 50.5 | 61.67 | 2.78 | 29.9 | — | |
| RAG (BM25)LLM Backbone=Gemma-4-26B, Memory Category=No Structure, Augmentation Strategy=with GOAL-MEM2026.05 | 50.3 | 64 | 8.67 | 34.3 | — | |
| A-MEMLLM Backbone=Gemma-4-26B, Memory Category=Graph-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 49.7 | 64 | 7.67 | 33.1 | — | |
| RAG (Dense)LLM Backbone=Gemma-4-26B, Memory Category=No Structure, Augmentation Strategy=with GOAL-MEM2026.05 | 49.6 | 63 | 9.67 | 33.1 | — | |
| A-MEMLLM Backbone=GPT-5.4-mini, Memory Category=Graph-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 49.3 | 59.17 | 6.67 | 29 | — | |
| RAG (BM25)LLM Backbone=GPT-5.4-mini, Memory Category=No Structure, Augmentation Strategy=with GOAL-MEM2026.05 | 49.1 | 57.78 | 3.33 | 28 | — | |
| RAPTORLLM Backbone=GPT-5.4-mini, Memory Category=Tree-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 48.7 | 56.67 | 6.67 | 27.1 | — | |
| Mem0LLM Backbone=GPT-5.4-mini, Memory Category=No Structure, Augmentation Strategy=with GOAL-MEM2026.05 | 48.4 | 59.17 | 5 | 28.2 | — | |
| GraphRAGLLM Backbone=Gemma-4-26B, Memory Category=Graph-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 47.2 | 61.67 | 8.67 | 31.2 | — | |
| Mem0LLM Backbone=Gemma-4-26B, Memory Category=No Structure, Augmentation Strategy=with GOAL-MEM2026.05 | 45.7 | 58.67 | 9 | 25.7 | — | |
| MemTreeLLM Backbone=GPT-5.4-mini, Memory Category=Tree-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 43.6 | 55.83 | 4.16 | 22.8 | — | |
| GraphRAGLLM Backbone=GPT-5.4-mini, Memory Category=Graph-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 43.1 | 50.83 | 4.5 | 23.4 | — | |
| TMEMModel=Qwen3-8B2026.06 | 41.87 | — | — | — | 25.42 | |
| TMEMModel=Qwen3-4B2026.06 | 41.24 | — | — | — | 25.54 | |
| Mem0-GraphLLM Backbone=GPT-5.4-mini, Memory Category=Graph-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 41 | 52.16 | 3.83 | 21.7 | — | |
| Mem0-GraphLLM Backbone=Gemma-4-26B, Memory Category=Graph-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 40.9 | 51 | 6.33 | 27.1 | — | |
| MemAgent/MEM1Model=Qwen3-4B2026.06 | 36.45 | — | — | — | 23.2 | |
| MemAgent/MEM1Model=Qwen3-8B2026.06 | 31.66 | — | — | — | 17.4 | |
| AMEMModel=Qwen3-4B2026.06 | 29.22 | — | — | — | 17.76 | |
| AMEMModel=Qwen3-8B2026.06 | 28.99 | — | — | — | 18.4 | |
| RAPTORLLM Backbone=GPT-5.4-mini, Memory Category=Tree-Based, Augmentation Strategy=Backbone Alone2026.05 | 21.6 | 50 | — | — | — | |
| RAG (BM25)LLM Backbone=GPT-5.4-mini, Memory Category=No Structure, Augmentation Strategy=Backbone Alone2026.05 | 21 | 54.44 | — | — | — | |
| MemTreeLLM Backbone=GPT-5.4-mini, Memory Category=Tree-Based, Augmentation Strategy=Backbone Alone2026.05 | 20.8 | 51.67 | — | — | — | |
| RAG (Dense)LLM Backbone=GPT-5.4-mini, Memory Category=No Structure, Augmentation Strategy=Backbone Alone2026.05 | 20.5 | 58.89 | — | — | — | |
| RAPTORLLM Backbone=Gemma-4-26B, Memory Category=Tree-Based, Augmentation Strategy=Backbone Alone2026.05 | 20.4 | 51.11 | — | — | — | |
| A-MEMLLM Backbone=GPT-5.4-mini, Memory Category=Graph-Based, Augmentation Strategy=Backbone Alone2026.05 | 20.4 | 52.5 | — | — | — | |
| Mem0LLM Backbone=GPT-5.4-mini, Memory Category=No Structure, Augmentation Strategy=Backbone Alone2026.05 | 20.2 | 54.17 | — | — | — | |
| Mem0LLM Backbone=Gemma-4-26B, Memory Category=No Structure, Augmentation Strategy=Backbone Alone2026.05 | 20 | 49.67 | — | — | — | |
| GraphRAGLLM Backbone=GPT-5.4-mini, Memory Category=Graph-Based, Augmentation Strategy=Backbone Alone2026.05 | 19.8 | 46.33 | — | — | — | |
| Mem0-GraphLLM Backbone=GPT-5.4-mini, Memory Category=Graph-Based, Augmentation Strategy=Backbone Alone2026.05 | 19.4 | 48.33 | — | — | — | |
| A-MEMLLM Backbone=Gemma-4-26B, Memory Category=Graph-Based, Augmentation Strategy=Backbone Alone2026.05 | 16.6 | 56.33 | — | — | — | |
| RAG (Dense)LLM Backbone=Gemma-4-26B, Memory Category=No Structure, Augmentation Strategy=Backbone Alone2026.05 | 16.5 | 53.33 | — | — | — | |
| RAG (BM25)LLM Backbone=Gemma-4-26B, Memory Category=No Structure, Augmentation Strategy=Backbone Alone2026.05 | 16 | 55.33 | — | — | — | |
| GraphRAGLLM Backbone=Gemma-4-26B, Memory Category=Graph-Based, Augmentation Strategy=Backbone Alone2026.05 | 16 | 53 | — | — | — | |
| MemTreeLLM Backbone=Gemma-4-26B, Memory Category=Tree-Based, Augmentation Strategy=Backbone Alone2026.05 | 15.5 | 56 | — | — | — | |
| Mem0-GraphLLM Backbone=Gemma-4-26B, Memory Category=Graph-Based, Augmentation Strategy=Backbone Alone2026.05 | 13.8 | 44.67 | — | — | — | |
| No MemoryModel=Qwen3-4B2026.06 | 5.3 | — | — | — | 0 | |
| No MemoryModel=Qwen3-8B2026.06 | 3.5 | — | — | — | 0 |