Memory-Augmented Dialogue on PersonaMem v1.0 (test)
74.53Overall ScorePersonaTree-ALL
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| PersonaTree-ALLResponse Model=DeepSeek-R1-0528, Extraction Model=DeepSeek-V3.12026.01 | 74.53 | 81.4 | 70.91 | 26.88 | 92.93 | 82.01 | 87.72 | 82.35 | — | |
| PersonaTree-ALLResponse Model=Longcat-Flash-Chat, Extraction Model=DeepSeek-V3.12026.01 | 73.34 | 88.37 | 69.09 | 31.18 | 91.92 | 69.78 | 87.72 | 76.47 | — | |
| FluxMem2026.02 | 72.43 | 85.27 | 70.91 | 36.56 | 91.92 | 65.47 | — | — | 82.46 | |
| PersonaTree-ALLResponse Model=DeepSeek-V3.1, Extraction Model=DeepSeek-V3.12026.01 | 70.8 | 88.37 | 63.64 | 24.73 | 86.87 | 68.35 | 91.23 | 70.59 | — | |
| ALL DialogueResponse Model=DeepSeek-R1-0528, Context=Full Context2026.01 | 64.86 | 69.77 | 63.64 | 11.83 | 84.85 | 73.38 | 84.21 | 70.59 | — | |
| MemoryOSResponse Model=Longcat-Flash-Chat, Extraction Model=DeepSeek-V3.12026.01 | 64.52 | 79.84 | 65.45 | 16.13 | 87.88 | 64.75 | 66.67 | 64.71 | — | |
| ALL DialogueResponse Model=DeepSeek-V3.1, Context=Full Context2026.01 | 63.84 | 78.29 | 61.82 | 12.9 | 81.82 | 65.47 | 77.19 | 76.47 | — | |
| O-Mem2026.02 | 63.25 | 67.81 | 65.45 | 21.51 | 89.9 | 61.15 | — | — | 73.68 | |
| MemoryOSResponse Model=DeepSeek-V3.1, Extraction Model=DeepSeek-V3.12026.01 | 61.97 | 75.19 | 67.27 | 9.68 | 80.81 | 63.31 | 73.68 | 70.59 | — | |
| MemoryOSResponse Model=DeepSeek-R1-0528, Extraction Model=DeepSeek-V3.12026.01 | 61.97 | 64.34 | 76.36 | 11.83 | 82.83 | 66.19 | 73.68 | 76.47 | — | |
| Mem0Response Model=DeepSeek-V3.1, Extraction Model=DeepSeek-V3.12026.01 | 61.8 | 82.95 | 47.27 | 7.53 | 83.84 | 63.31 | 71.93 | 70.59 | — | |
| ALL DialogueResponse Model=Longcat-Flash-Chat, Context=Full Context2026.01 | 61.8 | 79.07 | 63.64 | 15.05 | 87.88 | 53.96 | 68.42 | 70.59 | — | |
| A-MemResponse Model=DeepSeek-V3.1, Extraction Model=DeepSeek-V3.12026.01 | 60.61 | 76.74 | 43.64 | 6.45 | 85.86 | 64.03 | 71.93 | 76.47 | — | |
| A-MemResponse Model=Longcat-Flash-Chat, Extraction Model=DeepSeek-V3.12026.01 | 60.44 | 80.62 | 50.91 | 8.6 | 85.86 | 66.19 | 47.37 | 70.59 | — | |
| Mem0Response Model=Longcat-Flash-Chat, Extraction Model=DeepSeek-V3.12026.01 | 60.27 | 77.52 | 54.55 | 9.68 | 85.86 | 64.03 | 50.88 | 76.47 | — | |
| Memory OS2026.02 | 59.97 | 72.72 | 72.72 | 17.2 | 78.79 | 58.27 | — | — | 56.14 | |
| A-Mem2026.02 | 59.75 | 63.01 | 69.09 | 27.96 | 85.86 | 54.68 | — | — | 57.89 | |
| LangMemResponse Model=Longcat-Flash-Chat, Extraction Model=DeepSeek-V3.12026.01 | 57.39 | 75.97 | 50.91 | 10.75 | 79.8 | 63.31 | 45.61 | 52.94 | — | |
| LangMemResponse Model=DeepSeek-R1-0528, Extraction Model=DeepSeek-V3.12026.01 | 56.54 | 62.79 | 54.55 | 18.28 | 78.79 | 66.19 | 49.12 | 41.18 | — | |
| LangMemResponse Model=DeepSeek-V3.1, Extraction Model=DeepSeek-V3.12026.01 | 56.37 | 72.87 | 52.73 | 9.68 | 78.79 | 61.15 | 50.88 | 47.06 | — | |
| Only LLMResponse Model=Longcat-Flash-Chat2026.01 | 54.33 | 62.02 | 43.64 | 8.6 | 84.85 | 71.22 | 28.07 | 52.94 | — | |
| Only LLMResponse Model=DeepSeek-V3.12026.01 | 52.63 | 59.69 | 43.64 | 5.38 | 80.81 | 65.47 | 42.11 | 52.94 | — | |
| HippoRAG 22026.02 | 51.87 | 76.74 | 23.64 | 33.33 | 78.79 | 56.12 | — | — | 38.6 | |
| Mem02026.02 | 48.55 | 32.13 | 52.73 | 15.05 | 80.81 | 54.68 | — | — | 57.89 | |
| Mem0Response Model=DeepSeek-R1-0528, Extraction Model=DeepSeek-V3.12026.01 | 47.71 | 32.56 | 52.73 | 21.51 | 73.74 | 57.55 | 47.37 | 58.82 | — | |
| A-MemResponse Model=DeepSeek-R1-0528, Extraction Model=DeepSeek-V3.12026.01 | 47.03 | 27.13 | 45.45 | 27.96 | 76.77 | 55.4 | 43.86 | 76.47 | — | |
| Only LLMResponse Model=DeepSeek-R1-05282026.01 | 44.14 | 30.23 | 38.18 | 19.35 | 76.77 | 55.4 | 38.6 | 41.18 | — | |
| LangMem2026.02 | 40.64 | 31.29 | 40 | 24.73 | 81.82 | 53.24 | — | — | 8.77 |