Long-term Memory Evaluation on LongMemEval S (test)
94.4KU (Knowledge Update)EMem-G
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| EMem-GLLM=GPT-4o-mini2026.02 | 94.4 | 73.6 | 87.5 | 32.2 | 87 | 74.8 | 77.9 | — | — | — | |
| EMem-GLLM=GPT-4.1-mini2026.02 | 94.4 | 82.6 | 87.5 | 50 | 94.8 | 83.7 | 84.9 | — | — | — | |
| MnemisLLM=GPT-4.1-mini2026.02 | 93.6 | 86.5 | 100 | 100 | 98.6 | 86.5 | 91.6 | — | — | — | |
| MnemisLLM=GPT-4o-mini2026.02 | 92.3 | 76.7 | 100 | 90 | 97.1 | 83.5 | 87.2 | — | — | — | |
| TiMemAnswer Model=GPT-4o-2024-11-202026.01 | 87.69 | 72.78 | 85.71 | 55.33 | 96.28 | 73.38 | 78.96 | — | — | — | |
| Mem0LLM=GPT-4.1-mini2026.02 | 87.2 | 66.9 | 96.4 | 86.7 | 94.3 | 75.9 | 80.8 | — | — | — | |
| EverMemOSLLM=GPT-4.1-mini2026.02 | 87.2 | 78.5 | 78.6 | 96.7 | 100 | 71.2 | 82 | — | — | — | |
| A-MEMAnswer Model=GPT-4o-2024-11-202026.01 | 87.18 | 45.26 | 83.21 | 56.67 | 90 | 46.77 | 63.4 | — | — | — | |
| TiMemAnswer Model=GPT-4o-mini-2024-07-182026.01 | 86.16 | 70.83 | 82.14 | 63.33 | 95.71 | 68.42 | 76.88 | — | — | — | |
| Mem0Answer Model=GPT-4o-2024-11-202026.01 | 84.87 | 65.11 | 55 | 60.67 | 95.71 | 51.88 | 67.56 | — | — | — | |
| PREMemLLM=GPT-4.1-mini2026.02 | 84.6 | 57.1 | 12.5 | 36.7 | 92.9 | 59.4 | 60.8 | — | — | — | |
| RAGLLM=GPT-4.1-mini2026.02 | 80.8 | 54.9 | 94.6 | 86.7 | 82.9 | 67.7 | 72.6 | — | — | — | |
| NemoriLLM=GPT-4.1-mini2026.02 | 79.5 | 55.6 | 92.9 | 86.7 | 90 | 72.2 | 74.6 | — | — | — | |
| Mem0Answer Model=GPT-4o-mini-2024-07-182026.01 | 78.72 | 66.17 | 51.79 | 50 | 94.29 | 49.17 | 64.96 | — | — | — | |
| Full ContextLLM=GPT-4o-mini2026.02 | 78.2 | 38.3 | 89.3 | 6.7 | 78.6 | 42.1 | 55 | — | — | — | |
| Full ContextLLM=GPT-4.1-mini2026.02 | 76.9 | 51.1 | 98.2 | 16.7 | 85.7 | 60.2 | 65.6 | — | — | — | |
| MemOSAnswer Model=GPT-4o-mini-2024-07-182026.01 | 76.67 | 58.8 | 67.86 | 50.67 | 93.71 | 65.11 | 68.68 | — | — | — | |
| MemOSAnswer Model=GPT-4o-2024-11-202026.01 | 76.07 | 68.42 | 63.69 | 64.44 | 92.86 | 71.43 | 73.07 | — | — | — | |
| Mem0LLM=GPT-4o-mini2026.02 | 74.4 | 66.2 | 96.4 | 34 | 91.4 | 63.9 | 71.1 | — | — | — | |
| ZepLLM=GPT-4o-mini2026.02 | 74.4 | 47.4 | 75 | 53.3 | 92.9 | 54.1 | 63.2 | — | — | — | |
| A-MEMAnswer Model=GPT-4o-mini-2024-07-182026.01 | 72.82 | 40.3 | 87.5 | 39.33 | 82.86 | 36.09 | 55.44 | — | — | — | |
| RAGLLM=GPT-4o-mini2026.02 | 70.5 | 47.4 | 91.1 | 70 | 88.6 | 63.2 | 67.2 | — | — | — | |
| NemoriLLM=GPT-4o-mini2026.02 | 61.5 | 51.1 | 83.9 | 46.7 | 88.6 | 61.7 | 64.2 | — | — | — | |
| MemoryOSAnswer Model=GPT-4o-2024-11-202026.01 | 60 | 51.13 | 80 | 53.33 | 82.86 | 54.59 | 61.2 | — | — | — | |
| MemoryOSAnswer Model=GPT-4o-mini-2024-07-182026.01 | 56.15 | 44.81 | 78.18 | 51.33 | 81.14 | 53.38 | 58.04 | — | — | — | |
| MEMTIER + semanticRetrieval Method=BM25 + Semantic, Retrieval Strategy=DeepSeek-V4-Flash fact pre-population2026.05 | 43.6 | 18 | — | 6.7 | — | 31.6 | 38.2 | 68.6 | 73.2 | 41.2 | |
| MEMTIER BM25 onlyRetrieval Method=BM25 only, Retrieval Strategy=Raw episodic retrieval2026.05 | 34.6 | 6 | — | 0 | — | 20.3 | 25.2 | 54.3 | 46.4 | 14.2 | |
| MemoryBankAnswer Model=GPT-4o-2024-11-202026.01 | 22.56 | 12.78 | 61.43 | 13.33 | 33.43 | 13.53 | 22.88 | — | — | — | |
| MemoryBankAnswer Model=GPT-4o-mini-2024-07-182026.01 | 21.79 | 9.77 | 50 | 12 | 29.71 | 17.14 | 21.04 | — | — | — | |
| Full-ctx-7BBackbone=7B, Context Strategy=Full-context, Retrieval=No retrieval2026.05 | 0 | 0.8 | — | 0 | — | 10.5 | 5 | 5.7 | 10.7 | 5.4 |