# Agent Memory Retrieval on LoCoMo (test) Source: [Agent Memory Retrieval on LoCoMo (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:52:58.781Z Evaluation of Agent Memory Retrieval performance on the test split of the LoCoMo dataset, reporting detailed F1 and BLEU scores across single-hop, multi-hop, temporal, and open-domain scenarios, alongside average performance and token usage per query. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Agent Memory Retrieval | | Dataset (including any stated split) | LoCoMo (test) | | Metric | Multi-hop BLEU | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Multi-hop BLEU](): Higher is better; unit: PERCENT. - [Multi-hop F1](): Higher is better; unit: PERCENT. - [Temporal BLEU](): Higher is better; unit: PERCENT. - [Temporal F1](): Higher is better; unit: PERCENT. - [Open-domain BLEU](): Higher is better; unit: PERCENT. - [Open-domain F1](): Higher is better; unit: PERCENT. - [Single-hop BLEU](): Higher is better; unit: PERCENT. - [Single-hop F1](): Higher is better; unit: PERCENT. - [Average BLEU](): Higher is better; unit: PERCENT. - [Average F1](): Higher is better; unit: PERCENT. - [Tokens per Query](): Lower is better; unit: SCALAR. ## Recorded results | Method | Multi-hop BLEU | Result date | Report paper | | --- | --- | --- | --- | | xMemory | 27.56 | Not recorded | 6981819ea8b01e76cc8e2705 | | xMemory | 27.24 | Not recorded | 6981819ea8b01e76cc8e2705 | | LightMem | 25.33 | Not recorded | 6981819ea8b01e76cc8e2705 | | Nemori | 24.8 | Not recorded | 6981819ea8b01e76cc8e2705 | | Nemori | 24.2 | Not recorded | 6981819ea8b01e76cc8e2705 | | Naive RAG | 22.64 | Not recorded | 6981819ea8b01e76cc8e2705 | | xMemory | 22.21 | Not recorded | 6981819ea8b01e76cc8e2705 | | Naive RAG | 21.64 | Not recorded | 6981819ea8b01e76cc8e2705 | | LightMem | 19.67 | Not recorded | 6981819ea8b01e76cc8e2705 | | Nemori | 18.18 | Not recorded | 6981819ea8b01e76cc8e2705 | | MemoryOS | 17.12 | Not recorded | 6981819ea8b01e76cc8e2705 | | LightMem | 16.43 | Not recorded | 6981819ea8b01e76cc8e2705 | | A-Mem | 15.88 | Not recorded | 6981819ea8b01e76cc8e2705 | | A-Mem | 14.6 | Not recorded | 6981819ea8b01e76cc8e2705 | | MemoryOS | 14.43 | Not recorded | 6981819ea8b01e76cc8e2705 | | Naive RAG | 13.77 | Not recorded | 6981819ea8b01e76cc8e2705 | | MemoryOS | 13.26 | Not recorded | 6981819ea8b01e76cc8e2705 | | A-Mem | 9.2 | Not recorded | 6981819ea8b01e76cc8e2705 | ### xMemory Multi-hop BLEU: 27.56. Result ID: 699653b45698665ec498be1f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5 nano"}. This record does not assert independent reproduction. ### xMemory Multi-hop BLEU: 27.24. Result ID: 699653b45698665ec498be13. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### LightMem Multi-hop BLEU: 25.33. Result ID: 699653b45698665ec498be1c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5 nano"}. This record does not assert independent reproduction. ### Nemori Multi-hop BLEU: 24.8. Result ID: 699653b45698665ec498be1b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5 nano"}. This record does not assert independent reproduction. ### Nemori Multi-hop BLEU: 24.2. Result ID: 699653b45698665ec498be0f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### Naive RAG Multi-hop BLEU: 22.64. Result ID: 699653b45698665ec498be0e. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### xMemory Multi-hop BLEU: 22.21. Result ID: 699653b45698665ec498be19. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Ins"}. This record does not assert independent reproduction. ### Naive RAG Multi-hop BLEU: 21.64. Result ID: 699653b45698665ec498be1a. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"GPT-5 nano"}. This record does not assert independent reproduction. ### LightMem Multi-hop BLEU: 19.67. Result ID: 699653b45698665ec498be10. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### Nemori Multi-hop BLEU: 18.18. Result ID: 699653b45698665ec498be15. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Ins"}. This record does not assert independent reproduction. ### MemoryOS Multi-hop BLEU: 17.12. Result ID: 699653b45698665ec498be12. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### LightMem Multi-hop BLEU: 16.43. Result ID: 699653b45698665ec498be16. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Ins"}. This record does not assert independent reproduction. ### A-Mem Multi-hop BLEU: 15.88. Result ID: 699653b45698665ec498be17. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Ins"}. This record does not assert independent reproduction. ### A-Mem Multi-hop BLEU: 14.6. Result ID: 699653b45698665ec498be1d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5 nano"}. This record does not assert independent reproduction. ### MemoryOS Multi-hop BLEU: 14.43. Result ID: 699653b45698665ec498be1e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-5 nano"}. This record does not assert independent reproduction. ### Naive RAG Multi-hop BLEU: 13.77. Result ID: 699653b45698665ec498be14. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Ins"}. This record does not assert independent reproduction. ### MemoryOS Multi-hop BLEU: 13.26. Result ID: 699653b45698665ec498be18. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Ins"}. This record does not assert independent reproduction. ### A-Mem Multi-hop BLEU: 9.2. Result ID: 699653b45698665ec498be11. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.