Long-horizon Question Answering on LongMemEval-RR
46.33F1 ScoreMemAgent-7B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MemAgent-7BQuery visible at write time?=Yes, Full raw-log access?=No, Source-evidence budget=native2026.06 | 46.33 | — | — | |
| TierMem with full raw logsQuery visible at write time?=No, Full raw-log access?=Yes, Source-evidence budget=full log2026.06 | 45.41 | 100 | 61.52 | |
| Oracle retentionQuery visible at write time?=No, Full raw-log access?=No, Source-evidence budget=81922026.06 | 44.99 | 99.98 | 83.67 | |
| Full raw-log RAGQuery visible at write time?=No, Full raw-log access?=Yes, Source-evidence budget=full log2026.06 | 36.45 | 100 | 60.82 | |
| EMBER-14B writer (Qwen2.5-14B)Query visible at write time?=No, Full raw-log access?=No, Source-evidence budget=81922026.06 | 30.17 | 32.15 | 31.12 | |
| EMBER-7B writer (Qwen2.5-7B)Query visible at write time?=No, Full raw-log access?=No, Source-evidence budget=81922026.06 | 27.68 | 29.66 | 29.15 | |
| TierMem with budgeted raw fallbackQuery visible at write time?=No, Full raw-log access?=No, Source-evidence budget=81922026.06 | 17.65 | 12.46 | 9.97 | |
| Source-snippet heuristicQuery visible at write time?=No, Full raw-log access?=No, Source-evidence budget=81922026.06 | 17.63 | 26.57 | 17.54 | |
| Memory-R1 (adapted)Query visible at write time?=No, Full raw-log access?=No, Source-evidence budget=81922026.06 | 15.65 | 17.14 | 14.64 | |
| Hybrid salienceQuery visible at write time?=No, Full raw-log access?=No, Source-evidence budget=81922026.06 | 15.38 | 21.4 | 17.63 | |
| MemAgent-7B (adapted)Query visible at write time?=No, Full raw-log access?=No, Source-evidence budget=81922026.06 | 13.11 | — | — | |
| Generated-query indexingQuery visible at write time?=No, Full raw-log access?=No, Source-evidence budget=81922026.06 | 12.05 | 20.21 | 14.23 | |
| Summary-onlyQuery visible at write time?=No, Full raw-log access?=No, Source-evidence budget=B-equiv.2026.06 | 11.56 | — | — | |
| Recency-BQuery visible at write time?=No, Full raw-log access?=No, Source-evidence budget=81922026.06 | 8.27 | 15.21 | 9.43 | |
| Reservoir-BQuery visible at write time?=No, Full raw-log access?=No, Source-evidence budget=81922026.06 | 4.98 | 7.53 | 6.15 | |
| Random-BQuery visible at write time?=No, Full raw-log access?=No, Source-evidence budget=81922026.06 | 3.5 | 6.12 | 4.25 |