Long-term dialogue memory on LongMemEval (test)
85.75AccuracyMemBuilder
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MemBuilderMemory construction model=Qwen3-4B, Training stage=SFT + RL, Answer model=Claude 4.5 Sonnet2026.01 | 85.75 | — | |
| MemBuilderMemory construction model=Claude 4.5 Sonnet, Answer model=Claude 4.5 Sonnet2026.01 | 85.5 | — | |
| MemBuilderMemory construction model=Qwen3-4B, Training stage=SFT, Answer model=Claude 4.5 Sonnet2026.01 | 84.25 | — | |
| MemBuilderMemory construction model=GPT-4.1, Answer model=Claude 4.5 Sonnet2026.01 | 78.5 | — | |
| MemBuilderMemory construction model=QwQ-32B, Answer model=Claude 4.5 Sonnet2026.01 | 76 | — | |
| PLUGMEMEmb=NVE, LLM=Q32 + Q72, #TokAvg.=362.582026.02 | 75.1 | 0.016 | |
| MIRIXMemory construction method=Prompting-based, Answer model=Claude 4.5 Sonnet2026.01 | 73.25 | — | |
| LiCoMemoryEmb=NVE, LLM=4o + Q72, #TokAvg.=5914.852026.02 | 73 | 0.0009 | |
| ZepEmb=BGE-m3, LLM=4o, #TokAvg.=16002026.02 | 71.2 | — | |
| RAG-UtteranceMemory construction method=Retrieval-based, Answer model=Claude 4.5 Sonnet2026.01 | 69 | — | |
| RAG-SessionMemory construction method=Retrieval-based, Answer model=Claude 4.5 Sonnet2026.01 | 66.75 | — | |
| Vanilla RetrievalEmb=NVE, LLM=Q72, #TokAvg.=3742.522026.02 | 63.6 | 0.0012 | |
| MemBuilderMemory construction model=Qwen3-4B, Training stage=RL, Answer model=Claude 4.5 Sonnet2026.01 | 62.75 | — | |
| All ContextEmb=-, LLM=Q72, #TokAvg.=107K2026.02 | 62.4 | 0 | |
| A-MemEmb=NVE, LLM=4o + Q72, #TokAvg.=4225.852026.02 | 61 | 0.001 | |
| MemBuilderMemory construction model=Qwen3-4B, Training stage=Base, Answer model=Claude 4.5 Sonnet2026.01 | 56 | — | |
| Mem0Memory construction method=Prompting-based, Answer model=Claude 4.5 Sonnet2026.01 | 47 | — | |
| No ContextEmb=-, LLM=Q72, #TokAvg.=-2026.02 | 14.8 | — |