Open-domain Question Answering on LoCoMo Open-Domain
32.6F1 ScoreLightMem
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LightMemBackbone=Claude-Haiku-4.52026.03 | 32.6 | 24.43 | 69.23 | |
| LightMemBackbone=GPT-4o-mini2026.03 | 31.85 | 24.23 | 76.92 | |
| MEMMA_LMBackbone=Claude-Haiku-4.52026.03 | 28.59 | 26.86 | 84.62 | |
| MEMMA_LMBackbone=GPT-4o-mini2026.03 | 24.58 | 22.44 | 76.92 | |
| AriadneMem2026.07 | 20.04 | 17.26 | 19.79 | |
| LangMemBackbone=Claude-Haiku-4.52026.03 | 20.01 | 20.85 | 69.23 | |
| Full TextBackbone=GPT-4o-mini2026.03 | 18.25 | 19.56 | 61.54 | |
| Full TextBackbone=Claude-Haiku-4.52026.03 | 18.25 | 19.56 | 61.54 | |
| A-MemBackbone=GPT-4o-mini2026.03 | 18.18 | 15.27 | 53.85 | |
| Naive RAGBackbone=GPT-4o-mini2026.03 | 17.4 | 16.65 | 46.15 | |
| Naive RAGBackbone=Claude-Haiku-4.52026.03 | 17.4 | 16.65 | 46.15 | |
| A-MemBackbone=Claude-Haiku-4.52026.03 | 16.34 | 17.76 | 38.46 | |
| LangMemBackbone=GPT-4o-mini2026.03 | 14.91 | 14.03 | 38.46 | |
| M+2026.07 | 9.63 | 6.99 | 19.79 | |
| MemoryLLM2026.07 | 9.53 | 6.84 | 22.92 | |
| InsideOutA-TMA augmentation=true2026.07 | 9.51 | 7.2 | 27.08 | |
| Graphiti/ZepA-TMA augmentation=false2026.07 | 9.41 | 7.57 | 33.33 | |
| Graphiti/ZepA-TMA augmentation=true2026.07 | 9.06 | 6.46 | 39.58 | |
| InsideOutA-TMA augmentation=false2026.07 | 9.02 | 7.25 | 32.29 | |
| Mem02026.07 | 6.86 | 5.29 | 42.71 | |
| SeCom2026.07 | 6.8 | 4.86 | 20.83 | |
| AtomMem2026.07 | 6.37 | 4.21 | 19.79 | |
| A-MemA-TMA augmentation=true2026.07 | 3.85 | 2.07 | 42.86 | |
| A-MemA-TMA augmentation=false2026.07 | 2.29 | 0.81 | 33.33 |