Long-term dialogue memory on LoCoMo (test)
84.23AccuracyMemBuilder
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| MemBuilderMemory construction model=Qwen3-4B, Training stage=SFT + RL, Answer model=Claude 4.5 Sonnet, Memory construction method=Training-based2026.01 | 84.23 | — | — | — | — | — | — | — | |
| MemBuilderMemory construction model=Claude 4.5 Sonnet, Memory construction method=Prompting-based, Answer model=Claude 4.5 Sonnet2026.01 | 82.61 | — | — | — | — | — | — | — | |
| MemBuilderMemory construction model=Qwen3-4B, Training stage=SFT, Answer model=Claude 4.5 Sonnet, Memory construction method=Training-based2026.01 | 81.74 | — | — | — | — | — | — | — | |
| MemBuilderMemory construction model=GPT-4.1, Memory construction method=Prompting-based, Answer model=Claude 4.5 Sonnet2026.01 | 79.91 | — | — | — | — | — | — | — | |
| MemBuilderMemory construction model=Qwen3-4B, Training stage=RL, Answer model=Claude 4.5 Sonnet, Memory construction method=Training-based2026.01 | 79.31 | — | — | — | — | — | — | — | |
| MIRIXMemory construction method=Prompting-based, Answer model=Claude 4.5 Sonnet2026.01 | 77.48 | — | — | — | — | — | — | — | |
| MemBuilderMemory construction model=QwQ-32B, Memory construction method=Prompting-based, Answer model=Claude 4.5 Sonnet2026.01 | 77.47 | — | — | — | — | — | — | — | |
| MIRIXbaseline_status=true2026.02 | 77.37 | 80.14 | 76.01 | 67.71 | 78 | 317 | 78.96 | 880 | |
| StructMemBackbone=gpt-4o-mini, Embedding Model=text-embedding-3-small, Method Category=Structural Memory, Build Tokens (In)=1.501, Build Tokens (Out)=0.436, Build Tokens (Sum)=1.937, Calls=1056, Time (s)=228542026.04 | 76.82 | 81.09 | 68.77 | 46.88 | 81.62 | — | — | — | |
| MMAvariant=st, components=S + T, consensus_module=disabled2026.02 | 75.94 | 79.08 | 67.91 | 61.46 | 79.55 | 298 | 79.64 | 883.6 | |
| MemobaseBackbone=gpt-4o-mini, Embedding Model=text-embedding-3-small, Method Category=Structural Memory2026.04 | 75.78 | 77.17 | 70.92 | 46.88 | 85.05 | — | — | — | |
| ZepBackbone=gpt-4o-mini, Embedding Model=text-embedding-3-small, Method Category=Structural Memory2026.04 | 75.14 | 79.79 | 74.11 | 66.04 | 67.71 | — | — | — | |
| RAG-UtteranceMemory construction method=Retrieval-based, Answer model=Claude 4.5 Sonnet, Retrieval granularity=Utterance-level2026.01 | 74.87 | — | — | — | — | — | — | — | |
| FullContextBackbone=gpt-4o-mini, Embedding Model=text-embedding-3-small, Method Category=RAG2026.04 | 73.83 | 86.56 | 68.79 | 56.25 | 50.16 | — | — | — | |
| MMAcomponents=S + T + Ccon, variant=Full Model2026.02 | 72.31 | 73.76 | 62.31 | 59.38 | 77.05 | 335 | 76.8 | 793.6 | |
| OpenAIBackbone=gpt-4o-mini, Embedding Model=text-embedding-3-small, Method Category=RAG2026.04 | 71.82 | 84.66 | 69.86 | 53.12 | 45.48 | — | — | — | |
| RAG-SessionMemory construction method=Retrieval-based, Answer model=Claude 4.5 Sonnet, Retrieval granularity=Session-level2026.01 | 70.35 | — | — | — | — | — | — | — | |
| LightRAGBackbone=gpt-4o-mini, Embedding Model=text-embedding-3-small, Method Category=RAG, Build Tokens (In)=10.014, Build Tokens (Out)=1.916, Build Tokens (Sum)=11.931, Calls=13576, Time (s)=604692026.04 | 68.83 | 77.53 | 66.31 | 50 | 53.89 | — | — | — | |
| Mem0gBackbone=gpt-4o-mini, Embedding Model=text-embedding-3-small, Method Category=Structural Memory, Build Tokens (In)=33.512, Build Tokens (Out)=2.313, Build Tokens (Sum)=35.825, Calls=53514, Time (s)=1156702026.04 | 68.44 | 75.71 | 65.71 | 47.19 | 58.13 | — | — | — | |
| MemBuilderMemory construction model=Qwen3-4B, Training stage=Base, Answer model=Claude 4.5 Sonnet, Memory construction method=Training-based2026.01 | 68.07 | — | — | — | — | — | — | — | |
| Mem0Backbone=gpt-4o-mini, Embedding Model=text-embedding-3-small, Method Category=Flat Memory, Build Tokens (In)=10.958, Build Tokens (Out)=1.239, Build Tokens (Sum)=12.196, Calls=9181, Time (s)=300572026.04 | 66.88 | 72.93 | 67.13 | 51.15 | 59.19 | — | — | — | |
| A-MemBackbone=gpt-4o-mini, Embedding Model=text-embedding-3-small, Method Category=Flat Memory, Build Tokens (In)=9.126, Build Tokens (Out)=2.368, Build Tokens (Sum)=11.494, Calls=11754, Time (s)=606072026.04 | 64.16 | 72.06 | 56.03 | 31.25 | 60.44 | — | — | — | |
| MiniRAGBackbone=gpt-4o-mini, Embedding Model=text-embedding-3-small, Method Category=RAG, Build Tokens (In)=9.022, Build Tokens (Out)=1.081, Build Tokens (Sum)=10.103, Calls=2508, Time (s)=25662026.04 | 63.51 | 75.74 | 56.74 | 58.33 | 38.94 | — | — | — | |
| Memory-R1Memory construction model=Llama-3.1-8B-Instruct, Answer model=Llama-3.1-8B-Instruct, Memory construction method=Training-based2026.01 | 62.74 | — | — | — | — | — | — | — | |
| MemoryOSBackbone=gpt-4o-mini, Embedding Model=text-embedding-3-small, Method Category=Structural Memory, Build Tokens (In)=1.889, Build Tokens (Out)=0.939, Build Tokens (Sum)=2.868, Calls=5534, Time (s)=242202026.04 | 58.25 | 67.06 | 56.74 | 45.83 | 40.19 | — | — | — | |
| LangMemBackbone=gpt-4o-mini, Embedding Model=text-embedding-3-small, Method Category=Flat Memory, Build Tokens (In)=9.873, Build Tokens (Out)=1.192, Build Tokens (Sum)=11.066, Calls=5990, Time (s)=262812026.04 | 58.1 | 71.12 | 62.23 | 47.92 | 23.43 | — | — | — | |
| Mem0Memory construction method=Prompting-based, Answer model=Claude 4.5 Sonnet2026.01 | 51.64 | — | — | — | — | — | — | — | |
| Flat MemoryContext=Paradigm comparison study, Backbone=gpt-4o-mini, Embedding Model=text-embedding-3-small2026.04 | — | 78.83 | 66.31 | 46.88 | 78.5 | — | — | — | |
| Graph MemoryContext=Paradigm comparison study, Backbone=gpt-4o-mini, Embedding Model=text-embedding-3-small2026.04 | — | 80.5 | 66.67 | 48.96 | 76.64 | — | — | — | |
| StructMem (w/o Cross-Event)Context=Ablation study, Backbone=gpt-4o-mini, Embedding Model=text-embedding-3-small2026.04 | — | 80.86 | 66.31 | 46.88 | 79.44 | — | — | — |