LLM Agent Memory Retrieval on LoCoMo v1 (full)
35.7F1 (Multi-Hop)SYNAPSE
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SYNAPSEBackbone=GPT-4o-mini2026.01 | 35.7 | 26.2 | 50.1 | 44.5 | 25.9 | 19.2 | 48.9 | 42.9 | 96.6 | 96.4 | 40.5 | 32.6 | 1 | |
| ZepBackbone=GPT-4o-mini2026.01 | 35.5 | 25.8 | 48.5 | 40.2 | 23.1 | 18 | 48 | 41.5 | 65.4 | 64 | 39.7 | 31.2 | 2.6 | |
| MemoryOSBackbone=GPT-4o-mini2026.01 | 35.3 | 25.2 | 41.2 | 30.8 | 20 | 16.5 | 48.6 | 43 | — | — | 38 | 29.1 | — | |
| LangMemBackbone=GPT-4o-mini2026.01 | 34.5 | 23.7 | 30.8 | 25.8 | 24.3 | 19.2 | 40.9 | 33.6 | 47.6 | 46.3 | 34.3 | 25.7 | 5 | |
| AriGraphBackbone=GPT-4o-mini2026.01 | 28.5 | 21 | 43.2 | 33.5 | 14.5 | 13 | 45.1 | 38 | 48.5 | 47 | 33.7 | 26.2 | 4.6 | |
| A-MemBackbone=GPT-4o-mini2026.01 | 27 | 20.1 | 45.9 | 36.7 | 12.1 | 12 | 44.7 | 37.1 | 50 | 49.5 | 33.3 | 26.2 | 4.8 | |
| MemGPTBackbone=GPT-4o-mini2026.01 | 26.7 | 17.7 | 25.5 | 19.4 | 9.2 | 7.4 | 41 | 34.3 | 43.3 | 42.7 | 28 | 20.5 | 7.2 | |
| LoCoMoBackbone=GPT-4o-mini2026.01 | 25 | 19.8 | 18.4 | 14.8 | 12 | 11.2 | 40.4 | 29.1 | 69.2 | 68.8 | 25.6 | 19.9 | 7 | |
| ENGRAMBackbone=GPT-4o-mini2026.01 | 18.3 | 13.2 | 21.9 | 14.7 | 8.6 | 5.5 | 23.1 | 13.7 | 33.5 | 19.4 | 19.3 | 13.1 | 9.2 | |
| GraphRAGBackbone=GPT-4o-mini2026.01 | 16.5 | 11.8 | 22.4 | 15.2 | 10.1 | 8.4 | 24.5 | 18.2 | 15.2 | 12 | 18.3 | 14.2 | 8.8 | |
| ReadAgentBackbone=GPT-4o-mini2026.01 | 9.2 | 6.5 | 12.6 | 8.9 | 5.3 | 5.1 | 9.7 | 7.7 | 9.8 | 9 | 9.8 | 7.1 | 11 | |
| MemoryBankBackbone=GPT-4o-mini2026.01 | 5 | 4.8 | 9.7 | 7 | 5.6 | 5.9 | 6.6 | 5.2 | 7.4 | 6.5 | 6.3 | 5.4 | 11.6 |