Runtime Agent Memory on LoCoMo
43.05F1 ScoreBudgetMem-CAP
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BudgetMem-CAPBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 43.05 | 54.62 | 2.4 | |
| BudgetMem-CAPBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 41.22 | 53.18 | 0.61 | |
| BudgetMem-REABackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 40.92 | 52.23 | 2.9 | |
| BudgetMem-REABackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 40.19 | 53.34 | 1.11 | |
| BudgetMem-IMPBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 40.14 | 54.38 | 0.8 | |
| BudgetMem-IMPBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 38.75 | 50.32 | 1.8 | |
| MemoryOSBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 35.43 | 38.85 | 0.75 | |
| LightMemBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 33.88 | 40.76 | 1.5 | |
| LightMemBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 32.85 | 42.83 | 0.7 | |
| MemoryOSBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 30.62 | 34.55 | 1.97 | |
| A-MEMBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 27.65 | 38.54 | 2.88 | |
| A-MEMBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 26.43 | 32.96 | 2.88 | |
| MemoryBankBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 23.53 | 34.71 | 0.25 | |
| ReadAgentBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 22.48 | 31.05 | 0.57 | |
| ReadAgentBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 22.45 | 31.37 | 0.24 | |
| LangMemBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 22.31 | 25.96 | 0.48 | |
| MemoryBankBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 22.27 | 28.98 | 0.73 | |
| LangMemBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 20.89 | 23.4 | 0.14 | |
| Mem0Backbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 11.04 | 28.18 | 2.89 | |
| Mem0Backbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 10.77 | 25.32 | 1.15 |