Long-context reasoning and retrieval on LoCoMo (test)
95.12Single-Hop F1TraceMem
Evaluation Results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TraceMemBackbone=GPT-4.1-mini2026.02 | 95.12 | — | — | — | 89.36 | — | 90.97 | — | 80.21 | — | 92.27 | — | |
| TraceMemBackbone=GPT-4o-mini2026.02 | 93.1 | — | — | — | 92.2 | — | 86.6 | — | 70.83 | — | 90.19 | — | |
| FClatency p50=4.18s, latency p90=6.08s, latency p95=6.92s, latency p99=9.35s2026.01 | 92.2 | 86.1 | — | — | 82.6 | — | 76.6 | — | — | — | — | 75 | |
| FullTextBackbone=GPT-4.1-mini2026.02 | 92.03 | — | — | — | 87.23 | — | 81.62 | — | 66.67 | — | 87.4 | — | |
| EM+SMlatency p50=2.28s, latency p90=3.21s, latency p95=3.68s, latency p99=4.18s2026.01 | 89 | 87.7 | — | — | 84.5 | — | 90.4 | — | — | — | — | 76.1 | |
| NemoriBackbone=GPT-4.1-mini2026.02 | 87.04 | — | — | — | 74.82 | — | 76.32 | — | 54.17 | — | 80.52 | — | |
| EMlatency p50=2.19s, latency p90=2.94s, latency p95=3.23s, latency p99=3.84s2026.01 | 86.8 | 86.3 | — | — | 85.6 | — | 87.7 | — | — | — | — | 78.1 | |
| ReadAgentlatency p50=10.59s, latency p90=15.33s, latency p95=18.00s, latency p99=25.50s2026.01 | 85.7 | 79.8 | — | — | 68.8 | — | 75.4 | — | — | — | — | 75 | |
| FullTextBackbone=GPT-4o-mini2026.02 | 83.95 | — | — | — | 69.86 | — | 51.71 | — | 58.33 | — | 73.05 | — | |
| NemoriBackbone=GPT-4o-mini2026.02 | 82.52 | — | — | — | 63.12 | — | 68.54 | — | 50 | — | 74.03 | — | |
| LightMemBackbone=GPT-4.1-mini2026.02 | 81.21 | — | — | — | 70.57 | — | 78.82 | — | 47.92 | — | 76.69 | — | |
| LightMemBackbone=GPT-4o-mini2026.02 | 74.14 | — | — | — | 60.69 | — | 74.22 | — | 48.19 | — | 70.07 | — | |
| RAGlatency p50=1.50s, latency p90=2.15s, latency p95=2.40s, latency p99=3.17s2026.01 | 70 | 52.6 | — | — | 34.4 | — | 29.7 | — | — | — | — | 69.2 | |
| Mem0latency p50=1.04s, latency p90=1.63s, latency p95=1.81s, latency p99=2.07s2026.01 | 65.7 | 59.9 | — | — | 53.1 | — | 51.4 | — | — | — | — | 69.2 | |
| NaiveRAGBackbone=GPT-4.1-mini2026.02 | 62.07 | — | — | — | 39.01 | — | 37.69 | — | 47.92 | — | 51.88 | — | |
| A-MemBackbone=GPT-4.1-mini2026.02 | 61.47 | — | — | — | 56.03 | — | 63.55 | — | 39.58 | — | 59.55 | — | |
| E-memBackbone=Qwen2.5-14B2026.01 | 61.14 | 57.04 | 46.75 | 52.5 | 49.15 | 34.87 | 63.59 | 50.61 | 22.38 | 18.41 | — | — | |
| E-memBackbone=GPT-4o-mini2026.01 | 59.23 | 54.17 | 44.34 | 50.58 | 42.64 | 34.38 | 59.82 | 44.57 | 24.89 | 18.15 | — | — | |
| NaiveRAGBackbone=GPT-4o-mini2026.02 | 56.6 | — | — | — | 33.69 | — | 30.84 | — | 43.75 | — | 46.23 | — | |
| GAMBackbone=Qwen2.5-14B2026.01 | 56.35 | 50.41 | 43.48 | 51.07 | 38.94 | 28.55 | 53.76 | 50.01 | 20.84 | 15.09 | — | — | |
| HippoRAGlatency p50=3.51s, latency p90=6.01s, latency p95=7.00s, latency p99=9.72s2026.01 | 55.7 | 54.5 | — | — | 21.9 | — | 72.9 | — | — | — | — | 77.1 | |
| RAGBackbone=GPT-4o-mini2026.01 | 52.45 | 44.73 | 39.4 | 47.94 | 27.5 | 20.13 | 46.07 | 40.35 | 23.23 | 17.94 | — | — | |
| MEMORYOSBackbone=GPT-4o-mini2026.01 | 48.62 | 42.84 | 35.54 | 42.99 | 35.27 | 25.22 | 41.15 | 30.76 | 20.02 | 16.52 | — | — | |
| A-MemBackbone=GPT-4o-mini2026.02 | 47.92 | — | — | — | 32.98 | — | 38.32 | — | 17.71 | — | 41.3 | — | |
| RAGBackbone=Qwen2.5-14B2026.01 | 47.87 | 38.27 | 33.07 | 42.89 | 26.38 | 19.54 | 30.78 | 25.97 | 14.16 | 10.52 | — | — | |
| GAMBackbone=GPT-4o-mini2026.01 | 47.74 | 45.31 | 37.78 | 40.9 | 34.84 | 27.72 | 53.91 | 43.93 | 26.03 | 19.48 | — | — | |
| Mem0Backbone=GPT-4o-mini2026.01 | 47.65 | 45.1 | 34.92 | 37.06 | 38.72 | 27.13 | 48.93 | 40.15 | 28.64 | 21.58 | — | — | |
| Long-ContextBackbone=GPT-4o-mini2026.01 | 46.68 | 37.31 | 29.57 | 37.54 | 29.23 | 22.76 | 25.97 | 19.42 | 16.87 | 13.7 | — | — | |
| MEMORYOSBackbone=Qwen2.5-14B2026.01 | 46.33 | 40.28 | 34.72 | 41.32 | 38.19 | 29.26 | 32.24 | 27.86 | 20.27 | 15.94 | — | — | |
| Long-ContextBackbone=Qwen2.5-14B2026.01 | 46.05 | 38.31 | 31.89 | 39.56 | 32.08 | 24.46 | 30.51 | 24.45 | 14.89 | 11.41 | — | — | |
| A-memBackbone=GPT-4o-mini2026.01 | 44.65 | 39.65 | 32.31 | 37.06 | 27.02 | 20.09 | 45.85 | 36.67 | 12.14 | 12.01 | — | — | |
| A-MEMlatency p50=0.77s, latency p90=1.06s, latency p95=1.25s, latency p99=2.09s2026.01 | 44.3 | 37.5 | — | — | 15.6 | — | 37.8 | — | — | — | — | 53.8 | |
| Mem0Backbone=Qwen2.5-14B2026.01 | 42.58 | 36.04 | 29.91 | 35.15 | 31.73 | 24.82 | 28.96 | 26.24 | 15.03 | 11.28 | — | — | |
| LIGHTMEMBackbone=GPT-4o-mini2026.01 | 41.79 | 38.44 | 34.37 | 37.83 | 29.78 | 24.9 | 43.71 | 39.72 | 16.89 | 13.92 | — | — | |
| LIGHTMEMBackbone=Qwen2.5-14B2026.01 | 34.92 | 31.39 | 27.15 | 31.22 | 25.45 | 19.61 | 32.03 | 27.7 | 15.81 | 11.81 | — | — | |
| A-memBackbone=Qwen2.5-14B2026.01 | 33.75 | 28.98 | 24.47 | 30.04 | 22.09 | 15.28 | 27.19 | 22.05 | 13.49 | 10.74 | — | — | |
| Zeplatency p50=1.25s, latency p90=1.65s, latency p95=1.74s, latency p99=2.27s2026.01 | 32.9 | 29.6 | — | — | 31.2 | — | 5.4 | — | — | — | — | 76.9 |