# Memory-augmented generation on LongMemEvalS (test) Source: [Memory-augmented generation on LongMemEvalS (test)]() Collection index: [llms.txt]() Data updated: 2026-07-02T07:33:29.423Z Evaluation of model performance on the test split of the LongMemEvalS dataset, focusing on memory-augmented generation capabilities assessed across single-session, multi-session, temporal reasoning, and knowledge update scenarios, reported via LLM and F1 scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Memory-augmented generation | | Dataset (including any stated split) | LongMemEvalS (test) | | Metric | User LLM (Single-Session) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 8 of 8 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [User LLM (Single-Session)](): Higher is better; unit: PERCENT. - [User F1 (Single-Session)](): Higher is better; unit: PERCENT. - [Preference LLM (Single-Session)](): Higher is better; unit: PERCENT. - [Preference F1 (Single-Session)](): Higher is better; unit: PERCENT. - [Assistant LLM (Single-Session)](): Higher is better; unit: PERCENT. - [Assistant F1 (Single-Session)](): Higher is better; unit: PERCENT. - [LLM (Multi-Session)](): Higher is better; unit: PERCENT. - [F1 (Multi-Session)](): Higher is better; unit: PERCENT. - [LLM (Temporal Reasoning)](): Higher is better; unit: PERCENT. - [F1 (Temporal Reasoning)](): Higher is better; unit: PERCENT. - [LLM (Knowledge Update)](): Higher is better; unit: PERCENT. - [F1 (Knowledge Update)](): Higher is better; unit: PERCENT. - [Overall LLM](): Higher is better; unit: PERCENT. - [Overall F1](): Higher is better; unit: PERCENT. ## Recorded results | Method | User LLM (Single-Session) | Result date | Report paper | | --- | --- | --- | --- | | Full-Context | 98.4 | Not recorded | 6a45f084dc71372d7795a209 | | TRACE | 93.8 | Not recorded | 6a45f084dc71372d7795a209 | | Nemori | 93.3 | Not recorded | 6a45f084dc71372d7795a209 | | A-Mem | 92.2 | Not recorded | 6a45f084dc71372d7795a209 | | TRACE | 79.7 | Not recorded | 6a45f084dc71372d7795a209 | | Full-Context | 78.1 | Not recorded | 6a45f084dc71372d7795a209 | | A-Mem | 78.1 | Not recorded | 6a45f084dc71372d7795a209 | | Nemori | 66.7 | Not recorded | 6a45f084dc71372d7795a209 | ### Full-Context User LLM (Single-Session): 98.4. Result ID: 6a46142c60b88bb3398422ad. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash"}. This record does not assert independent reproduction. ### TRACE User LLM (Single-Session): 93.8. Result ID: 6a46142c60b88bb3398422b0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash"}. This record does not assert independent reproduction. ### Nemori User LLM (Single-Session): 93.3. Result ID: 6a46142c60b88bb3398422ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-4o-mini"}. This record does not assert independent reproduction. ### A-Mem User LLM (Single-Session): 92.2. Result ID: 6a46142c60b88bb3398422ae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash"}. This record does not assert independent reproduction. ### TRACE User LLM (Single-Session): 79.7. Result ID: 6a46142c60b88bb3398422ac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-4o-mini"}. This record does not assert independent reproduction. ### Full-Context User LLM (Single-Session): 78.1. Result ID: 6a46142c60b88bb3398422a9. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"GPT-4o-mini"}. This record does not assert independent reproduction. ### A-Mem User LLM (Single-Session): 78.1. Result ID: 6a46142c60b88bb3398422aa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-4o-mini"}. This record does not assert independent reproduction. ### Nemori User LLM (Single-Session): 66.7. Result ID: 6a46142c60b88bb3398422af. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"DeepSeek-V4-Flash"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.