Runtime Agent Memory on HotpotQA
58.7F1 ScoreBudgetMem-CAP
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BudgetMem-CAPBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 58.7 | 72.08 | 0.22 | |
| BudgetMem-REABackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 57.67 | 70.83 | 0.17 | |
| BudgetMem-CAPBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 53.87 | 64.85 | 0.93 | |
| BudgetMem-REABackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 51.12 | 61.93 | 0.99 | |
| BudgetMem-IMPBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 49.31 | 65.77 | 1.35 | |
| BudgetMem-IMPBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 46.67 | 57.42 | 0.63 | |
| LightMemBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 45.73 | 58.37 | 10.1 | |
| A-MEMBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 43.25 | 54.69 | 13.32 | |
| LightMemBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 41.29 | 55.42 | 8.56 | |
| MemoryOSBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 41.21 | 53.52 | 11.68 | |
| A-MEMBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 40.54 | 50.39 | 8.34 | |
| MemoryOSBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 34.5 | 43.36 | 13.32 | |
| Mem0Backbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 28.03 | 36.72 | 4.3 | |
| Mem0Backbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 24.72 | 37.89 | 2.02 | |
| LangMemBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 22.78 | 22.66 | 10.95 | |
| MemoryBankBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 22.25 | 23.75 | 7.75 | |
| LangMemBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 20.77 | 21.09 | 17.56 | |
| MemoryBankBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 18.64 | 31.25 | 1.79 | |
| ReadAgentBackbone=Qwen3-Next-80B-A3B-Instruct, Performance-first setting=true, Transfer evaluation=true2026.02 | 18.05 | 25.78 | 1.75 | |
| ReadAgentBackbone=LLaMA-3.3-70B-Instruct, Performance-first setting=true2026.02 | 15.33 | 30.08 | 4.19 |