Runtime Agent Memory on LongMemEval
40.53F1 ScoreBudgetMem-REA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BudgetMem-REABackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 40.53 | 58 | 0.67 | |
| BudgetMem-CAPBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 40.24 | 60.5 | 0.8 | |
| BudgetMem-IMPBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 37.47 | 56 | 0.71 | |
| BudgetMem-REABackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 35.84 | 59 | 0.26 | |
| BudgetMem-CAPBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 31.01 | 56 | 0.17 | |
| BudgetMem-IMPBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 29.18 | 52 | 0.3 | |
| ReadAgentBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 27.72 | 20.75 | 4.97 | |
| Mem0Backbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 27.7 | 42.08 | 13.57 | |
| LightMemBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 27.7 | 47.52 | 3.39 | |
| MemoryBankBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 26.74 | 32.67 | 3.94 | |
| LightMemBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 26.74 | 48.51 | 5.28 | |
| Mem0Backbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 25.71 | 36.14 | 4.96 | |
| A-MEMBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 21.74 | 33.17 | 80.02 | |
| ReadAgentBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 20.75 | 27.72 | 13.68 | |
| MemoryOSBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 13.35 | 33 | 15.84 | |
| MemoryOSBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 12.97 | 33.5 | 38.83 | |
| LangMemBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 12 | 17 | 16.6 | |
| LangMemBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 11.01 | 14 | 3.99 | |
| A-MEMBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 10.82 | 31.19 | 21 | |
| MemoryBankBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 10.56 | 28.22 | 3.45 |