Question Answering on LoCoMo Overall (BLEU-1, F1, LLM-as-a-Judge)
39.81BLEU-1DeltaMem-4o-mini
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DeltaMem-4o-miniTraining Protocol=Training-free, Memory Manager Architecture=Agentic, Backbone=GPT-4o-mini2026.04 | 39.81 | 50.32 | 72.92 | |
| DeltaMem-8B-RLTraining Protocol=RL-trained, Memory Manager Architecture=Agentic, Backbone=Qwen3-8B2026.04 | 39.78 | 50.72 | 75.13 | |
| DeltaMem-4B-RLTraining Protocol=RL-trained, Memory Manager Architecture=Agentic, Backbone=Qwen3-4B2026.04 | 39.29 | 50.13 | 74.03 | |
| DeltaMem-4BTraining Protocol=Training-free, Memory Manager Architecture=Agentic, Backbone=Qwen3-4B2026.04 | 37.73 | 47.51 | 70.13 | |
| DeltaMem-8BTraining Protocol=Training-free, Memory Manager Architecture=Agentic, Backbone=Qwen3-8B2026.04 | 36.63 | 47.27 | 71.04 | |
| LightMemTraining Protocol=Training-free, Memory Manager Architecture=Multi-Agent2026.04 | 34.17 | 44.81 | 66.43 | |
| Mem0Training Protocol=Training-free, Memory Manager Architecture=Multi-Agent2026.04 | 33.79 | 42.55 | 57.86 | |
| ZepTraining Protocol=Training-free, Memory Manager Architecture=Multi-Agent2026.04 | 33.64 | 43.57 | 65.99 | |
| LangMemTraining Protocol=Training-free, Memory Manager Architecture=Multi-Agent2026.04 | 30.06 | 36.88 | 58.11 | |
| Memory-R1-7BTraining Protocol=RL-trained, Memory Manager Architecture=Multi-Agent, Backbone=7B2026.04 | 26.06 | 33.64 | 62.34 | |
| A-MemTraining Protocol=Training-free, Memory Manager Architecture=Multi-Agent2026.04 | 24.82 | 29.96 | 48.38 |