Long-context memory-based conversation reasoning on BEAM 500K (test)
57Pass@0.5SelfMem
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SelfMemScale=500K2026.07 | 57 | 0.487 | 1.828 | 33.8 | 3,559 | 0 | |
| RAGScale=500K2026.07 | 42.1 | 0.346 | 1.258 | 0.6 | 2,821 | 37,011 | |
| LoCoMoScale=500K2026.07 | 36.9 | 0.302 | 8.156 | 0.6 | 2,812 | 1,400 | |
| Mem0Scale=500K2026.07 | 36.4 | 0.303 | 9.366 | 36.4 | 6,846 | 13,898 | |
| ReadAgentScale=500K2026.07 | 35.9 | 0.286 | 7.926 | 0.8 | 2,799 | 1,050 | |
| A-MemScale=500K2026.07 | 34.9 | 0.281 | 6.5 | 0.6 | 2,812 | 1,400 | |
| MemoryBankScale=500K2026.07 | 33.9 | 0.276 | 1.335 | 0.5 | 2,799 | 39,897 | |
| MemGPTScale=500K2026.07 | 30.7 | 0.249 | 11.917 | 1.2 | 2,799 | 980 | |
| CompressionScale=500K2026.07 | 25.3 | 0.195 | 8.144 | 71.4 | 4,066 | 0 |