Multi-Choice Question Answering on RefMem-Bench
59.4AccuracyREMIND
Evaluation results
| Method | Links | ||
|---|---|---|---|
| REMINDBackbone=Qwen3-VL-8B2026.05 | 59.4 | 58.1 | |
| LightMemBackbone=GPT-4o-mini2026.05 | 48.3 | 48.5 | |
| GAMBackbone=GPT-4o-mini2026.05 | 47 | 46.2 | |
| A-MemBackbone=GPT-4o-mini2026.05 | 46.3 | 48.5 | |
| Mem0Backbone=GPT-4o-mini2026.05 | 42.5 | — | |
| MemoryOSBackbone=GPT-4o-mini2026.05 | 41.2 | 48.9 | |
| LangMemBackbone=GPT-4o-mini2026.05 | 41.2 | — | |
| MemGPTBackbone=GPT-4o-mini2026.05 | 38.4 | 41.4 | |
| Qwen3-VL-30BBackbone=30B2026.05 | 36.3 | 46 | |
| - w/ NaiveRAGBackbone=Qwen3-VL-8B2026.05 | 33.6 | 45.9 | |
| Base ModelBackbone=Qwen3-VL-8B2026.05 | 33.2 | 45 | |
| MemAgent-7BBackbone=7B2026.05 | 25 | 40.6 | |
| DS-Distill-Qwen-14BBackbone=14B2026.05 | 24.6 | 43.8 | |
| MEM1-7BBackbone=7B2026.05 | 19.8 | 33.9 |