# Agent Memory Question Answering on PerLTQA (test) Source: [Agent Memory Question Answering on PerLTQA (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:37:01.009Z Evaluation of model performance on the test split of the PerLTQA dataset specifically for Agent Memory Question Answering tasks, measured using BLEU, F1, ROUGE-L, and query token count. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Agent Memory Question Answering | | Dataset (including any stated split) | PerLTQA (test) | | Metric | BLEU | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [BLEU](): Higher is better; unit: PERCENT. - [F1 Score](): Higher is better; unit: PERCENT. - [ROUGE-L](): Higher is better; unit: PERCENT. - [Query Tokens](): Lower is better; unit: SCALAR. ## Recorded results | Method | BLEU | Result date | Report paper | | --- | --- | --- | --- | | xMemory | 42.68 | Not recorded | 6981819ea8b01e76cc8e2705 | | Nemori | 41.01 | Not recorded | 6981819ea8b01e76cc8e2705 | | xMemory | 36.79 | Not recorded | 6981819ea8b01e76cc8e2705 | | xMemory | 36.24 | Not recorded | 6981819ea8b01e76cc8e2705 | | MemoryOS | 35.14 | Not recorded | 6981819ea8b01e76cc8e2705 | | MemoryOS | 35.14 | Not recorded | 6981819ea8b01e76cc8e2705 | | Naive RAG | 33.67 | Not recorded | 6981819ea8b01e76cc8e2705 | | Nemori | 33.44 | Not recorded | 6981819ea8b01e76cc8e2705 | | A-Mem | 33.12 | Not recorded | 6981819ea8b01e76cc8e2705 | | Nemori | 32.55 | Not recorded | 6981819ea8b01e76cc8e2705 | | Naive RAG | 32.08 | Not recorded | 6981819ea8b01e76cc8e2705 | | A-Mem | 28.62 | Not recorded | 6981819ea8b01e76cc8e2705 | | MemoryOS | 27.66 | Not recorded | 6981819ea8b01e76cc8e2705 | | Naive RAG | 27.35 | Not recorded | 6981819ea8b01e76cc8e2705 | | LightMem | 23.47 | Not recorded | 6981819ea8b01e76cc8e2705 | | LightMem | 16.97 | Not recorded | 6981819ea8b01e76cc8e2705 | | LightMem | 14.04 | Not recorded | 6981819ea8b01e76cc8e2705 | | A-Mem | 11.4 | Not recorded | 6981819ea8b01e76cc8e2705 | ### xMemory BLEU: 42.68. Result ID: 699653b45698665ec498be07. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Ins"}. This record does not assert independent reproduction. ### Nemori BLEU: 41.01. Result ID: 699653b45698665ec498be03. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Ins"}. This record does not assert independent reproduction. ### xMemory BLEU: 36.79. Result ID: 699653b45698665ec498be0d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-5 nano"}. This record does not assert independent reproduction. ### xMemory BLEU: 36.24. Result ID: 699653b45698665ec498be01. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-8B"}. This record does not assert independent reproduction. ### MemoryOS BLEU: 35.14. Result ID: 699653b45698665ec498be00. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-8B"}. This record does not assert independent reproduction. ### MemoryOS BLEU: 35.14. Result ID: 699653b45698665ec498be06. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Ins"}. This record does not assert independent reproduction. ### Naive RAG BLEU: 33.67. Result ID: 699653b45698665ec498be02. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Ins"}. This record does not assert independent reproduction. ### Nemori BLEU: 33.44. Result ID: 699653b45698665ec498be09. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-5 nano"}. This record does not assert independent reproduction. ### A-Mem BLEU: 33.12. Result ID: 699653b45698665ec498be0b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-5 nano"}. This record does not assert independent reproduction. ### Nemori BLEU: 32.55. Result ID: 699653b45698665ec498bdfd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-8B"}. This record does not assert independent reproduction. ### Naive RAG BLEU: 32.08. Result ID: 699653b45698665ec498bdfc. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen3-8B"}. This record does not assert independent reproduction. ### A-Mem BLEU: 28.62. Result ID: 699653b45698665ec498bdff. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-8B"}. This record does not assert independent reproduction. ### MemoryOS BLEU: 27.66. Result ID: 699653b45698665ec498be0c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-5 nano"}. This record does not assert independent reproduction. ### Naive RAG BLEU: 27.35. Result ID: 699653b45698665ec498be08. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-5 nano"}. This record does not assert independent reproduction. ### LightMem BLEU: 23.47. Result ID: 699653b45698665ec498be04. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Ins"}. This record does not assert independent reproduction. ### LightMem BLEU: 16.97. Result ID: 699653b45698665ec498bdfe. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-8B"}. This record does not assert independent reproduction. ### LightMem BLEU: 14.04. Result ID: 699653b45698665ec498be0a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-5 nano"}. This record does not assert independent reproduction. ### A-Mem BLEU: 11.4. Result ID: 699653b45698665ec498be05. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Ins"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.