Lifelong Memory Question Answering on LoCoMo
40.36MultiHop BLEUTriMem
Evaluation Results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TriMemModel=GPT-4o2026.05 | 40.36 | 46 | 51.39 | 60.41 | 39.27 | 50.15 | 40.61 | 47.78 | 42.73 | 50.23 | 1,272 | |
| LightMemModel=GPT-4o2026.05 | 35.3 | 45.16 | 43.6 | 58.57 | 10.56 | 23.2 | 36.72 | 46.6 | 36.26 | 47.37 | 677 | |
| TriMemModel=GPT-4.1-mini2026.05 | 35.2 | 42.59 | 49.56 | 64.72 | 36.86 | 43.88 | 45.25 | 55.36 | 43.79 | 54.26 | 1,217 | |
| TriMemModel=GPT-5-nano2026.05 | 34.86 | 45.25 | 42.45 | 57.05 | 33.55 | 40.52 | 54.26 | 62.88 | 46.96 | 57.04 | 1,256 | |
| LightMemModel=GPT-4.1-mini2026.05 | 32.93 | 40.33 | 47.53 | 55.23 | 18.31 | 21.91 | 37.68 | 48.39 | 37.66 | 46.69 | 695 | |
| SimpleMemModel=GPT-4.1-mini2026.05 | 32.4 | 39.33 | 43.69 | 58.01 | 19.56 | 24.5 | 43.41 | 53.99 | 39.97 | 50.3 | 587 | |
| SimpleMemModel=GPT-4o2026.05 | 31.34 | 35.58 | 35.78 | 46.96 | 18.96 | 17.01 | 37.11 | 43.94 | 34.64 | 41.36 | 627 | |
| Mem0Model=GPT-4.1-mini2026.05 | 28.81 | 31.44 | 35.41 | 46.24 | 18.51 | 17.93 | 31.25 | 35.34 | 30.88 | 35.81 | 1,153 | |
| LightMemModel=GPT-5-nano2026.05 | 28.63 | 38.21 | 39.72 | 55.51 | 18.79 | 22.74 | 31.19 | 42.01 | 31.73 | 42.93 | 723 | |
| Mem0Model=GPT-4o2026.05 | 25.52 | 32.36 | 32.48 | 42.7 | 14.5 | 18.5 | 30.02 | 39.84 | 28.74 | 37.74 | 1,195 | |
| SimpleMemModel=GPT-5-nano2026.05 | 25.42 | 33.28 | 32.15 | 45.75 | 20.77 | 24.31 | 39.65 | 46.71 | 34.3 | 42.65 | 655 | |
| Mem0Model=GPT-5-nano2026.05 | 22.55 | 28.58 | 35.52 | 48.82 | 18.33 | 16.75 | 28.99 | 35.65 | 28.51 | 35.92 | 1,074 | |
| MemoryOSModel=GPT-4o2026.05 | 22.52 | 31.76 | 38.31 | 47.08 | 12.91 | 18.06 | 38.26 | 43.67 | 33.81 | 40.6 | 944 | |
| A-MemModel=GPT-4o2026.05 | 20.9 | 26.12 | 35.39 | 48.64 | 10.74 | 12.33 | 37.11 | 42.08 | 32.14 | 38.67 | 1,152 | |
| LoCoMoModel=GPT-5-nano2026.05 | 20.45 | 19.04 | 12.69 | 16.56 | 13.83 | 20.85 | 13.5 | 15.23 | 14.62 | 16.56 | 16,863 | |
| LoCoMoModel=GPT-4o2026.05 | 19.64 | 19.2 | 9.5 | 13.95 | 11.87 | 16.6 | 13.81 | 16.12 | 13.86 | 16.26 | 16,863 | |
| MemoryOSModel=GPT-4.1-mini2026.05 | 16.46 | 24.02 | 34.78 | 46.52 | 14.89 | 19.58 | 36.18 | 43.92 | 30.95 | 39.3 | 936 | |
| A-MemModel=GPT-5-nano2026.05 | 15.54 | 20.11 | 27.23 | 32.43 | 10.86 | 12.55 | 27.26 | 31.91 | 24.09 | 28.65 | 1,175 | |
| A-MemModel=GPT-4.1-mini2026.05 | 15.11 | 20.66 | 41.57 | 50.94 | 11.18 | 13.2 | 38.25 | 43.72 | 33.01 | 39.1 | 1,276 | |
| Naïve RAGModel=GPT-4o2026.05 | 14.36 | 15.35 | 11.48 | 16.17 | 9.03 | 9.09 | 26.67 | 35.03 | 20.15 | 25.88 | 1,119 | |
| Naïve RAGModel=GPT-4.1-mini2026.05 | 11.49 | 13.24 | 20.52 | 28.8 | 11.79 | 10.75 | 22.85 | 30.29 | 19.59 | 25.64 | 1,119 | |
| MemoryOSModel=GPT-5-nano2026.05 | 10.74 | 23.5 | 32.5 | 39.71 | 10.02 | 20.3 | 34.28 | 40.34 | 28.09 | 35.88 | 952 | |
| Naïve RAGModel=GPT-5-nano2026.05 | 10.13 | 13.29 | 8.78 | 13.09 | 9.25 | 12.24 | 20.29 | 28.44 | 15.34 | 21.46 | 1,119 | |
| LoCoMoModel=GPT-4.1-mini2026.05 | 8 | 17.26 | 10.17 | 14.89 | 8.29 | 16.28 | 17.43 | 19.36 | 13.62 | 17.85 | 16,863 |