Long-term memory evaluation on LongMemEval S
97.14Single-User ScoreMemoryOS
Evaluation Results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| MemoryOSBackbone=GPT-4o-mini2026.05 | 97.14 | 89.29 | 70 | 48.87 | 73.08 | 58.65 | 67.8 | 705.34 | 8.89 | — | |
| Mem0Backbone=GPT-4o-mini2026.05 | 95.71 | 55.36 | 70 | 54.14 | 69.23 | 57.89 | 64 | 1,244.87 | 1.9 | — | |
| Mem0Backbone=GPT-4.1-mini2026.05 | 95.71 | 50 | 90 | 62.41 | 74.36 | 69.92 | 71.2 | 1,626.54 | 2 | — | |
| A-MemBackbone=GPT-4.1-mini2026.05 | 95.71 | 100 | 63.33 | 52.63 | 82.05 | 61.65 | 71.6 | 1,264.25 | 15.46 | — | |
| RecMemBackbone=GPT-4.1-mini2026.05 | 95.71 | 96.43 | 86.67 | 68.42 | 75.64 | 65.41 | 76.8 | 365.49 | 5.89 | — | |
| MemoryOSBackbone=GPT-4.1-mini2026.05 | 94.29 | 89.29 | 100 | 54.89 | 80.77 | 67.67 | 74.4 | 669.22 | 9.19 | — | |
| A-MemBackbone=GPT-4o-mini2026.05 | 92.54 | 98.21 | 36.67 | 44.36 | 70.51 | 53.38 | 63.8 | 1,180.23 | 13.1 | — | |
| RecMemBackbone=GPT-4o-mini2026.05 | 91.43 | 85.71 | 46.67 | 61.65 | 80.77 | 56.39 | 69.2 | 329.55 | 5.63 | — | |
| FullContextBackbone=GPT-4.1-mini2026.05 | 91.43 | 100 | 56.67 | 48.12 | 75.64 | 53.38 | 66.2 | 0 | 112.25 | — | |
| LightMem2026.05 | 88.6 | — | 76.7 | 85.7 | 92.3 | 47.4 | 75.2 | — | — | — | |
| Mem02026.05 | 87.1 | — | 63.5 | 40.6 | 69.2 | 50.4 | 57.4 | — | — | — | |
| Naive RAGBackbone=GPT-4o-mini2026.05 | 85.71 | 83.93 | 46.67 | 41.35 | 65.38 | 52.63 | 59.4 | 0 | 11.92 | — | |
| Naive RAGBackbone=GPT-4.1-mini2026.05 | 85.71 | 82.36 | 83.33 | 45.86 | 75.64 | 63.91 | 67 | 0 | 11.93 | — | |
| SimpleMem2026.05 | 85.7 | — | 76.7 | 83.5 | 79.5 | 60.9 | 75.9 | — | — | — | |
| PPROGRPO Training=false2026.05 | 81.4 | — | 80 | 89.3 | 80.8 | 74.4 | 81.5 | — | — | — | |
| Full-context2026.05 | 47.1 | — | 60 | 27.1 | 41 | 30.1 | 35.8 | — | — | — | |
| FullContextBackbone=GPT-4o-mini2026.05 | 44.29 | 80.36 | 53.33 | 34.59 | 55.13 | 35.34 | 45.6 | 0 | 112.3 | — | |
| DCPMfull (S1+S2)Extraction/Inference Model Pair=deepseek-v4-flash2026.06 | — | — | — | — | — | — | — | — | — | 85.17 | |
| DCPMfull (S1+S2)Extraction/Inference Model Pair=kimi-2.52026.06 | — | — | — | — | — | — | — | — | — | 86.14 | |
| DCPMlite (S1)Extraction/Inference Model Pair=deepseek-v4-flash2026.06 | — | — | — | — | — | — | — | — | — | 84.73 | |
| DCPMlite (S1)Extraction/Inference Model Pair=kimi-2.52026.06 | — | — | — | — | — | — | — | — | — | 85.81 | |
| Long-Ctx AgentExtraction/Inference Model Pair=deepseek-v4-flash2026.06 | — | — | — | — | — | — | — | — | — | 88.43 | |
| Long-Ctx AgentExtraction/Inference Model Pair=kimi-2.52026.06 | — | — | — | — | — | — | — | — | — | 80.61 | |
| Mem0Extraction/Inference Model Pair=deepseek-v4-flash2026.06 | — | — | — | — | — | — | — | — | — | 56.84 | |
| Mem0Extraction/Inference Model Pair=kimi-2.52026.06 | — | — | — | — | — | — | — | — | — | 48.87 | |
| Mem0 + graphExtraction/Inference Model Pair=deepseek-v4-flash2026.06 | — | — | — | — | — | — | — | — | — | 57.14 | |
| Mem0 + graphExtraction/Inference Model Pair=kimi-2.52026.06 | — | — | — | — | — | — | — | — | — | 51.26 | |
| Zep / GraphitiExtraction/Inference Model Pair=deepseek-v4-flash2026.06 | — | — | — | — | — | — | — | — | — | 67.83 | |
| Zep / GraphitiExtraction/Inference Model Pair=kimi-2.52026.06 | — | — | — | — | — | — | — | — | — | 62.35 |