# Long-term memory question answering on LoCoMo 0.6.0 Source: [Long-term memory question answering on LoCoMo 0.6.0]() Collection index: [llms.txt]() Data updated: 2026-05-07T09:23:02.082Z Evaluation of Long-term memory question answering performance on the LoCoMo dataset version 0.6.0, reporting Accuracy, statistical confidence intervals, and cost per correct answer. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Long-term memory question answering | | Dataset (including any stated split) | LoCoMo 0.6.0 | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 10 of 10 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [Accuracy 95% CI](): Higher is better; unit: PERCENT. - [Correct Count](): Higher is better; unit: SCALAR. - [Cost per Correct Answer](): Lower is better; unit: OTHER. - [Chi-Squared Statistic](): Lower is better; unit: SCALAR. - [P-value](): Lower is better; unit: SCALAR. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | EverMemOS | 94.48 | Not recorded | 69fc1c8d5497b3b670c884f7 | | TM Pro | 93 | Not recorded | 69fc1c8d5497b3b670c884f7 | | TM Base | 92.01 | Not recorded | 69fc1c8d5497b3b670c884f7 | | TM Edge | 89.65 | Not recorded | 69fc1c8d5497b3b670c884f7 | | RAG (ChromaDB) | 86.17 | Not recorded | 69fc1c8d5497b3b670c884f7 | | Engram | 84.55 | Not recorded | 69fc1c8d5497b3b670c884f7 | | BM25 | 80.45 | Not recorded | 69fc1c8d5497b3b670c884f7 | | Zep (published) | 71 | Not recorded | 69fc1c8d5497b3b670c884f7 | | Supermemory | 65.39 | Not recorded | 69fc1c8d5497b3b670c884f7 | | Mem0 | 61.43 | Not recorded | 69fc1c8d5497b3b670c884f7 | ### EverMemOS Accuracy: 94.48. Result ID: 69fc59e047f34ce924df58a9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Answer model":"gpt-4.1-mini","Judge model":"gpt-4o-mini","Judge runs / question":"3 (majority)","Answer temperature":"0","Judge temperature":"0","Max answer tokens":"200","Retrieval top-k":"100","Retrieval infra":"proprietary cloud infrastructure (4-billion-parameter GPU-hosted embedder)"}. This record does not assert independent reproduction. ### TM Pro Accuracy: 93. Result ID: 69fc59e047f34ce924df58aa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Answer model":"gpt-4.1-mini","Judge model":"gpt-4o-mini","Judge runs / question":"3 (majority)","Answer temperature":"0","Judge temperature":"0","Max answer tokens":"200","Retrieval top-k":"100","3-run means":"true"}. This record does not assert independent reproduction. ### TM Base Accuracy: 92.01. Result ID: 69fc59e047f34ce924df58ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Answer model":"gpt-4.1-mini","Judge model":"gpt-4o-mini","Judge runs / question":"3 (majority)","Answer temperature":"0","Judge temperature":"0","Max answer tokens":"200","Retrieval top-k":"100","3-run means":"true"}. This record does not assert independent reproduction. ### TM Edge Accuracy: 89.65. Result ID: 69fc59e047f34ce924df58ac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Answer model":"gpt-4.1-mini","Judge model":"gpt-4o-mini","Judge runs / question":"3 (majority)","Answer temperature":"0","Judge temperature":"0","Max answer tokens":"200","Retrieval top-k":"100","3-run means":"true"}. This record does not assert independent reproduction. ### RAG (ChromaDB) Accuracy: 86.17. Result ID: 69fc59e047f34ce924df58ad. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Answer model":"gpt-4.1-mini","Judge model":"gpt-4o-mini","Judge runs / question":"3 (majority)","Answer temperature":"0","Judge temperature":"0","Max answer tokens":"200","Retrieval top-k":"100"}. This record does not assert independent reproduction. ### Engram Accuracy: 84.55. Result ID: 69fc59e047f34ce924df58ae. [Reporting paper]() Recorded evaluation configuration: {"Answer model":"gpt-4.1-mini","Judge model":"gpt-4o-mini","Judge runs / question":"3 (majority)","Answer temperature":"0","Judge temperature":"0","Max answer tokens":"200","Retrieval top-k":"100"}. This record does not assert independent reproduction. ### BM25 Accuracy: 80.45. Result ID: 69fc59e047f34ce924df58af. [Reporting paper]() Recorded evaluation configuration: {"Answer model":"gpt-4.1-mini","Judge model":"gpt-4o-mini","Judge runs / question":"3 (majority)","Answer temperature":"0","Judge temperature":"0","Max answer tokens":"200","Retrieval top-k":"100"}. This record does not assert independent reproduction. ### Zep (published) Accuracy: 71. Result ID: 69fc59e047f34ce924df58b0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"note":"using their own evaluation methodology, not the paper harness"}. This record does not assert independent reproduction. ### Supermemory Accuracy: 65.39. Result ID: 69fc59e047f34ce924df58b1. [Reporting paper]() Recorded evaluation configuration: {"Answer model":"gpt-4.1-mini","Judge model":"gpt-4o-mini","Judge runs / question":"3 (majority)","Answer temperature":"0","Judge temperature":"0","Max answer tokens":"200","Retrieval top-k":"100"}. This record does not assert independent reproduction. ### Mem0 Accuracy: 61.43. Result ID: 69fc59e047f34ce924df58b2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Answer model":"gpt-4.1-mini","Judge model":"gpt-4o-mini","Judge runs / question":"3 (majority)","Answer temperature":"0","Judge temperature":"0","Max answer tokens":"200","Retrieval top-k":"100"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.