Long-term memory question answering on LoCoMo 0.6.0
94.48AccuracyEverMemOS
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| EverMemOSAnswer model=gpt-4.1-mini, Judge model=gpt-4o-mini, Judge runs / question=3 (majority), Answer temperature=0, Judge temperature=0, Max answer tokens=200, Retrieval top-k=100, Retrieval infra=proprietary cloud infrastructure (4-billion-parameter GPU-hosted embedder)2026.05 | 94.48 | 93.23 | 1,455 | 0.001 | — | — | |
| TM ProAnswer model=gpt-4.1-mini, Judge model=gpt-4o-mini, Judge runs / question=3 (majority), Answer temperature=0, Judge temperature=0, Max answer tokens=200, Retrieval top-k=100, 3-run means=true2026.05 | 93 | 91.6 | 1,432 | 0.0013 | 4.09 | 0.043 | |
| TM BaseAnswer model=gpt-4.1-mini, Judge model=gpt-4o-mini, Judge runs / question=3 (majority), Answer temperature=0, Judge temperature=0, Max answer tokens=200, Retrieval top-k=100, 3-run means=true2026.05 | 92.01 | 90.54 | 1,417 | 0.0011 | 1.12 | 0.29 | |
| TM EdgeAnswer model=gpt-4.1-mini, Judge model=gpt-4o-mini, Judge runs / question=3 (majority), Answer temperature=0, Judge temperature=0, Max answer tokens=200, Retrieval top-k=100, 3-run means=true2026.05 | 89.65 | 88.05 | 1,381 | 0.001 | 6.54 | 0.011 | |
| RAG (ChromaDB)Answer model=gpt-4.1-mini, Judge model=gpt-4o-mini, Judge runs / question=3 (majority), Answer temperature=0, Judge temperature=0, Max answer tokens=200, Retrieval top-k=1002026.05 | 86.17 | 84.35 | 1,327 | 0.0011 | 11.56 | 6.7 | |
| EngramAnswer model=gpt-4.1-mini, Judge model=gpt-4o-mini, Judge runs / question=3 (majority), Answer temperature=0, Judge temperature=0, Max answer tokens=200, Retrieval top-k=1002026.05 | 84.55 | 82.65 | 1,302 | 0.0011 | 2.28 | 0.13 | |
| BM25Answer model=gpt-4.1-mini, Judge model=gpt-4o-mini, Judge runs / question=3 (majority), Answer temperature=0, Judge temperature=0, Max answer tokens=200, Retrieval top-k=1002026.05 | 80.45 | 78.4 | 1,239 | 0.0011 | 17.39 | 3 | |
| Zep (published)note=using their own evaluation methodology, not the paper harness2026.05 | 71 | — | — | — | — | — | |
| SupermemoryAnswer model=gpt-4.1-mini, Judge model=gpt-4o-mini, Judge runs / question=3 (majority), Answer temperature=0, Judge temperature=0, Max answer tokens=200, Retrieval top-k=1002026.05 | 65.39 | 62.98 | 1,007 | 0.0019 | 98.09 | 10 | |
| Mem0Answer model=gpt-4.1-mini, Judge model=gpt-4o-mini, Judge runs / question=3 (majority), Answer temperature=0, Judge temperature=0, Max answer tokens=200, Retrieval top-k=1002026.05 | 61.43 | 58.97 | 946 | 0.0031 | 6.26 | 0.012 |