Memory extraction on LOCOMO (main)
85.37Single Hop ScoreMemReader-4B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| MemReader-4BOptimization=GRPO2026.04 | 85.37 | 81.44 | 75.8 | 65.62 | 81.42 | 49.45 | 1,950 | |
| MemReader-0.6B2026.04 | 84.7 | 76.95 | 76.22 | 53.4 | 79.56 | 52.54 | 1,976 | |
| MemOSBackbone=4o-mini2026.04 | 84.06 | 73.16 | 75.9 | 57.29 | 78.7 | 51.9 | 1,854 | |
| MemReader-4BOptimization=SFT2026.04 | 81.88 | 76.12 | 71.02 | 62.15 | 77.33 | 47.77 | 784 | |
| MemU2026.04 | 74.91 | 72.34 | 43.61 | 54.17 | 66.67 | — | 4,000 | |
| Mem02026.04 | 68.97 | 61.7 | 58.26 | 50 | 64.2 | — | 1,000 | |
| MemoryOS2026.04 | 67.3 | 59.34 | 42.26 | 59.03 | 60.11 | — | 5,500 |