# Accurate Retrieval (AR) suite Source: [Accurate Retrieval (AR) suite]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:39:51.231Z Evaluation of model performance using the Accurate Retrieval (AR) suite, reporting scores across Convo, Hotpot, LoCo, LME, MSC, Perl, and SQuAD components. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Accurate Retrieval | | Dataset (including any stated split) | Accurate Retrieval (AR) suite | | Metric | Convo Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 36 of 36 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Convo Score](): Higher is better; unit: PERCENT. - [Hotpot Score](): Higher is better; unit: PERCENT. - [LoCo Score](): Higher is better; unit: PERCENT. - [LME Score](): Higher is better; unit: PERCENT. - [MSC Score](): Higher is better; unit: PERCENT. - [Perl Score](): Higher is better; unit: PERCENT. - [SQuAD Score](): Higher is better; unit: PERCENT. - [Average Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Convo Score | Result date | Report paper | | --- | --- | --- | --- | | UMA | 881.7 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA | 866.1 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA | 854.9 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA | 854.9 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA | 837.1 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA | 799.1 | Not recorded | 699d308e6017c4460ffab3f4 | | MemAgent | 602.7 | Not recorded | 699d308e6017c4460ffab3f4 | | RAG | 513.4 | Not recorded | 699d308e6017c4460ffab3f4 | | MemAlpha | 84.8 | Not recorded | 699d308e6017c4460ffab3f4 | | Concat | 53.6 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA | 46.39 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA | 46.33 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA (2 Stage) | 44.4 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA | 44.36 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA | 44.21 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA (w/o Phase I) | 44.16 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA | 44.01 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA (Global Group) | 43.91 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA | 43.38 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA (w/o RL) | 43.28 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA | 40.94 | Not recorded | 699d308e6017c4460ffab3f4 | | UMA (w/o RL & Phase I) | 40.86 | Not recorded | 699d308e6017c4460ffab3f4 | | RAG | 38.66 | Not recorded | 699d308e6017c4460ffab3f4 | | RAG | 36.55 | Not recorded | 699d308e6017c4460ffab3f4 | | MemAgent | 33.55 | Not recorded | 699d308e6017c4460ffab3f4 | | MemAgent | 32.31 | Not recorded | 699d308e6017c4460ffab3f4 | | MemAgent-woq | 26.8 | Not recorded | 699d308e6017c4460ffab3f4 | | MemAlpha | 16.18 | Not recorded | 699d308e6017c4460ffab3f4 | | Mem1 | 15.6 | Not recorded | 699d308e6017c4460ffab3f4 | | MemAlpha | 14.33 | Not recorded | 699d308e6017c4460ffab3f4 | | Mem1 | 14.22 | Not recorded | 699d308e6017c4460ffab3f4 | | MemAgent-woq | 12.13 | Not recorded | 699d308e6017c4460ffab3f4 | | Mem1 | 11.63 | Not recorded | 699d308e6017c4460ffab3f4 | | MemAgent-woq | 10.8 | Not recorded | 699d308e6017c4460ffab3f4 | | Concat | 8.65 | Not recorded | 699d308e6017c4460ffab3f4 | | Concat | 8.63 | Not recorded | 699d308e6017c4460ffab3f4 | ### UMA Convo Score: 881.7. Result ID: 699e23f55698665ec4de190f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### UMA Convo Score: 866.1. Result ID: 699e23f55698665ec4de190b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Phase I (sequential memory maintenance)":false}. This record does not assert independent reproduction. ### UMA Convo Score: 854.9. Result ID: 699e23f55698665ec4de190c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RL training":false}. This record does not assert independent reproduction. ### UMA Convo Score: 854.9. Result ID: 699e23f55698665ec4de190e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"training stages":"2 Stage"}. This record does not assert independent reproduction. ### UMA Convo Score: 837.1. Result ID: 699e23f55698665ec4de190d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"task grouping strategy":"Global Group"}. This record does not assert independent reproduction. ### UMA Convo Score: 799.1. Result ID: 699e23f55698665ec4de190a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"RL training":false,"Phase I (sequential memory maintenance)":false}. This record does not assert independent reproduction. ### MemAgent Convo Score: 602.7. Result ID: 699e23f55698665ec4de1906. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### RAG Convo Score: 513.4. Result ID: 699e23f55698665ec4de1905. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"k":20}. This record does not assert independent reproduction. ### MemAlpha Convo Score: 84.8. Result ID: 699e23f55698665ec4de1909. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Concat Convo Score: 53.6. Result ID: 699e23f55698665ec4de1904. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### UMA Convo Score: 46.39. Result ID: 699e23f55698665ec4de18f7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### UMA Convo Score: 46.33. Result ID: 699e23f55698665ec4de18d3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Is Proposed Full Method":true}. This record does not assert independent reproduction. ### UMA (2 Stage) Convo Score: 44.4. Result ID: 699e23f55698665ec4de18f6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Processing Stages":2}. This record does not assert independent reproduction. ### UMA Convo Score: 44.36. Result ID: 699e23f55698665ec4de18d2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Stages":2}. This record does not assert independent reproduction. ### UMA Convo Score: 44.21. Result ID: 699e23f55698665ec4de18cf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Phase I":false}. This record does not assert independent reproduction. ### UMA (w/o Phase I) Convo Score: 44.16. Result ID: 699e23f55698665ec4de18f3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Phase I":false}. This record does not assert independent reproduction. ### UMA Convo Score: 44.01. Result ID: 699e23f55698665ec4de18d1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Grouping":"Global"}. This record does not assert independent reproduction. ### UMA (Global Group) Convo Score: 43.91. Result ID: 699e23f55698665ec4de18f5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Grouping Strategy":"Global"}. This record does not assert independent reproduction. ### UMA Convo Score: 43.38. Result ID: 699e23f55698665ec4de18d0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reinforcement Learning":false}. This record does not assert independent reproduction. ### UMA (w/o RL) Convo Score: 43.28. Result ID: 699e23f55698665ec4de18f4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reinforcement Learning":false}. This record does not assert independent reproduction. ### UMA Convo Score: 40.94. Result ID: 699e23f55698665ec4de18ce. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reinforcement Learning":false,"Phase I":false}. This record does not assert independent reproduction. ### UMA (w/o RL & Phase I) Convo Score: 40.86. Result ID: 699e23f55698665ec4de18f2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Reinforcement Learning":false,"Phase I":false}. This record does not assert independent reproduction. ### RAG Convo Score: 38.66. Result ID: 699e23f55698665ec4de18c9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"k":20}. This record does not assert independent reproduction. ### RAG Convo Score: 36.55. Result ID: 699e23f55698665ec4de18ed. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"k":20}. This record does not assert independent reproduction. ### MemAgent Convo Score: 33.55. Result ID: 699e23f55698665ec4de18ca. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### MemAgent Convo Score: 32.31. Result ID: 699e23f55698665ec4de18ee. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### MemAgent-woq Convo Score: 26.8. Result ID: 699e23f55698665ec4de1907. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### MemAlpha Convo Score: 16.18. Result ID: 699e23f55698665ec4de18cd. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Mem1 Convo Score: 15.6. Result ID: 699e23f55698665ec4de1908. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### MemAlpha Convo Score: 14.33. Result ID: 699e23f55698665ec4de18f1. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Mem1 Convo Score: 14.22. Result ID: 699e23f55698665ec4de18cc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### MemAgent-woq Convo Score: 12.13. Result ID: 699e23f55698665ec4de18cb. [Reporting paper]() Recorded evaluation configuration: {"without question":true}. This record does not assert independent reproduction. ### Mem1 Convo Score: 11.63. Result ID: 699e23f55698665ec4de18f0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### MemAgent-woq Convo Score: 10.8. Result ID: 699e23f55698665ec4de18ef. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Concat Convo Score: 8.65. Result ID: 699e23f55698665ec4de18c8. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Concat Convo Score: 8.63. Result ID: 699e23f55698665ec4de18ec. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.