Question Answering on LoCoMo excluding adversarial (test)
55.5Overall F1MemRouter
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MemRouterAnswer Agent / Backbone=Qwen3.5-35B-A3B, Backbone Transfer Setting=same router, no router retraining, Retrieval and Prompting Pipeline=Own system-level2026.05 | 55.5 | 60.9 | 56.1 | 49.7 | 23.8 | |
| MemRouterAnswer Agent / Backbone=Qwen2.5-7B-Instruct, Retrieval and Prompting Pipeline=Own system-level2026.05 | 52 | 57.5 | 52.4 | 44 | 27.1 | |
| Memory-R1-GRPOAnswer Agent / Backbone=Qwen2.5-7B-Instruct, Retrieval and Prompting Pipeline=Own system-level2026.05 | 43.1 | 33.6 | 23.6 | 47.8 | 46.9 | |
| MemoryOSAnswer Agent / Backbone=Qwen2.5-7B-Instruct, Retrieval and Prompting Pipeline=Own system-level2026.05 | 34.6 | 29.6 | 21 | 26.3 | 40.9 | |
| Mem0Answer Agent / Backbone=Qwen2.5-7B-Instruct, Retrieval and Prompting Pipeline=Own system-level2026.05 | 30.6 | 25 | 20.3 | 33.2 | 32.7 | |
| A-MemAnswer Agent / Backbone=Qwen2.5-7B-Instruct, Retrieval and Prompting Pipeline=Own system-level2026.05 | 26.1 | 19 | 14.7 | 23.7 | 30.6 | |
| LoCoMo (RAG)Answer Agent / Backbone=Qwen2.5-7B-Instruct, Retrieval and Prompting Pipeline=Own system-level2026.05 | 9 | 9.6 | 11.8 | 8.4 | 8.7 |