Direct-Answer Question Answering on RefMem-Bench
32.9AccuracyREMIND
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| REMINDBackbone=Qwen3-VL-8B2026.05 | 32.9 | 55.4 | 27.6 | 30.4 | |
| Qwen3-VL-30BBackbone=30B2026.05 | 23.6 | 40.4 | 19.5 | 22.4 | |
| LightMemBackbone=GPT-4o-mini2026.05 | 23.2 | 39.9 | 20.1 | 23.2 | |
| GAMBackbone=GPT-4o-mini2026.05 | 23.2 | 36.8 | 19.9 | 23 | |
| A-MemBackbone=GPT-4o-mini2026.05 | 22.9 | 39.6 | 16.4 | 18.5 | |
| Mem0Backbone=GPT-4o-mini2026.05 | 21.8 | — | 17.5 | 19.9 | |
| Base ModelBackbone=Qwen3-VL-8B2026.05 | 21.1 | 37.9 | 17.8 | 21.3 | |
| LangMemBackbone=GPT-4o-mini2026.05 | 21.1 | — | 19.9 | 22.3 | |
| - w/ NaiveRAGBackbone=Qwen3-VL-8B2026.05 | 19.8 | 38.7 | 17 | 19.8 | |
| DS-Distill-Qwen-14BBackbone=14B2026.05 | 19.1 | 39.1 | 15.4 | 18.8 | |
| MemoryOSBackbone=GPT-4o-mini2026.05 | 19 | 35.6 | 18.6 | 21.2 | |
| MemGPTBackbone=GPT-4o-mini2026.05 | 17.3 | 34.3 | 16.5 | 19.3 | |
| MEM1-7BBackbone=7B2026.05 | 12.7 | 24.6 | 10.7 | 12.9 | |
| MemAgent-7BBackbone=7B2026.05 | 12.6 | 29.7 | 12.1 | 14.2 |