# Long-horizon Question Answering on LongMemEval-RR Source: [Long-horizon Question Answering on LongMemEval-RR]() Collection index: [llms.txt]() Data updated: 2026-06-05T09:13:47.212Z Evaluation of long-horizon Question Answering capabilities on the LongMemEval-RR dataset, measured using Retain-Recall, Read-Recall, and F1 scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Long-horizon Question Answering | | Dataset (including any stated split) | LongMemEval-RR | | Metric | F1 Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [F1 Score](): Higher is better; unit: PERCENT. - [Retain-Recall](): Higher is better; unit: PERCENT. - [Read-Recall](): Higher is better; unit: PERCENT. ## Recorded results | Method | F1 Score | Result date | Report paper | | --- | --- | --- | --- | | MemAgent-7B | 46.33 | Not recorded | 6a22586e86173ad9c1468969 | | TierMem with full raw logs | 45.41 | Not recorded | 6a22586e86173ad9c1468969 | | Oracle retention | 44.99 | Not recorded | 6a22586e86173ad9c1468969 | | Full raw-log RAG | 36.45 | Not recorded | 6a22586e86173ad9c1468969 | | EMBER-14B writer (Qwen2.5-14B) | 30.17 | Not recorded | 6a22586e86173ad9c1468969 | | EMBER-7B writer (Qwen2.5-7B) | 27.68 | Not recorded | 6a22586e86173ad9c1468969 | | TierMem with budgeted raw fallback | 17.65 | Not recorded | 6a22586e86173ad9c1468969 | | Source-snippet heuristic | 17.63 | Not recorded | 6a22586e86173ad9c1468969 | | Memory-R1 (adapted) | 15.65 | Not recorded | 6a22586e86173ad9c1468969 | | Hybrid salience | 15.38 | Not recorded | 6a22586e86173ad9c1468969 | | MemAgent-7B (adapted) | 13.11 | Not recorded | 6a22586e86173ad9c1468969 | | Generated-query indexing | 12.05 | Not recorded | 6a22586e86173ad9c1468969 | | Summary-only | 11.56 | Not recorded | 6a22586e86173ad9c1468969 | | Recency-B | 8.27 | Not recorded | 6a22586e86173ad9c1468969 | | Reservoir-B | 4.98 | Not recorded | 6a22586e86173ad9c1468969 | | Random-B | 3.5 | Not recorded | 6a22586e86173ad9c1468969 | ### MemAgent-7B F1 Score: 46.33. Result ID: 6a22934afbc373592c20ef19. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Query visible at write time?":"Yes","Full raw-log access?":"No","Source-evidence budget":"native"}. This record does not assert independent reproduction. ### TierMem with full raw logs F1 Score: 45.41. Result ID: 6a22934afbc373592c20ef18. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Query visible at write time?":"No","Full raw-log access?":"Yes","Source-evidence budget":"full log"}. This record does not assert independent reproduction. ### Oracle retention F1 Score: 44.99. Result ID: 6a22934afbc373592c20ef1a. [Reporting paper]() Recorded evaluation configuration: {"Query visible at write time?":"No","Full raw-log access?":"No","Source-evidence budget":"8192"}. This record does not assert independent reproduction. ### Full raw-log RAG F1 Score: 36.45. Result ID: 6a22934afbc373592c20ef17. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Query visible at write time?":"No","Full raw-log access?":"Yes","Source-evidence budget":"full log"}. This record does not assert independent reproduction. ### EMBER-14B writer (Qwen2.5-14B) F1 Score: 30.17. Result ID: 6a22934afbc373592c20ef26. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Query visible at write time?":"No","Full raw-log access?":"No","Source-evidence budget":"8192"}. This record does not assert independent reproduction. ### EMBER-7B writer (Qwen2.5-7B) F1 Score: 27.68. Result ID: 6a22934afbc373592c20ef25. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Query visible at write time?":"No","Full raw-log access?":"No","Source-evidence budget":"8192"}. This record does not assert independent reproduction. ### TierMem with budgeted raw fallback F1 Score: 17.65. Result ID: 6a22934afbc373592c20ef23. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Query visible at write time?":"No","Full raw-log access?":"No","Source-evidence budget":"8192"}. This record does not assert independent reproduction. ### Source-snippet heuristic F1 Score: 17.63. Result ID: 6a22934afbc373592c20ef21. [Reporting paper]() Recorded evaluation configuration: {"Query visible at write time?":"No","Full raw-log access?":"No","Source-evidence budget":"8192"}. This record does not assert independent reproduction. ### Memory-R1 (adapted) F1 Score: 15.65. Result ID: 6a22934afbc373592c20ef24. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Query visible at write time?":"No","Full raw-log access?":"No","Source-evidence budget":"8192"}. This record does not assert independent reproduction. ### Hybrid salience F1 Score: 15.38. Result ID: 6a22934afbc373592c20ef1e. [Reporting paper]() Recorded evaluation configuration: {"Query visible at write time?":"No","Full raw-log access?":"No","Source-evidence budget":"8192"}. This record does not assert independent reproduction. ### MemAgent-7B (adapted) F1 Score: 13.11. Result ID: 6a22934afbc373592c20ef20. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Query visible at write time?":"No","Full raw-log access?":"No","Source-evidence budget":"8192"}. This record does not assert independent reproduction. ### Generated-query indexing F1 Score: 12.05. Result ID: 6a22934afbc373592c20ef22. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Query visible at write time?":"No","Full raw-log access?":"No","Source-evidence budget":"8192"}. This record does not assert independent reproduction. ### Summary-only F1 Score: 11.56. Result ID: 6a22934afbc373592c20ef1f. [Reporting paper]() Recorded evaluation configuration: {"Query visible at write time?":"No","Full raw-log access?":"No","Source-evidence budget":"B-equiv."}. This record does not assert independent reproduction. ### Recency-B F1 Score: 8.27. Result ID: 6a22934afbc373592c20ef1c. [Reporting paper]() Recorded evaluation configuration: {"Query visible at write time?":"No","Full raw-log access?":"No","Source-evidence budget":"8192"}. This record does not assert independent reproduction. ### Reservoir-B F1 Score: 4.98. Result ID: 6a22934afbc373592c20ef1d. [Reporting paper]() Recorded evaluation configuration: {"Query visible at write time?":"No","Full raw-log access?":"No","Source-evidence budget":"8192"}. This record does not assert independent reproduction. ### Random-B F1 Score: 3.5. Result ID: 6a22934afbc373592c20ef1b. [Reporting paper]() Recorded evaluation configuration: {"Query visible at write time?":"No","Full raw-log access?":"No","Source-evidence budget":"8192"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.