Long-context Question Answering on LoCoMo Overall
49.4F1 ScoreMEMMA_LM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MEMMA_LMBackbone=GPT-4o-mini2026.03 | 49.4 | 38.28 | 81.58 | |
| MEMMA_LMBackbone=Claude-Haiku-4.52026.03 | 45.1 | 36.53 | 76.97 | |
| LightMemBackbone=Claude-Haiku-4.52026.03 | 44.69 | 37.77 | 73.03 | |
| LightMemBackbone=GPT-4o-mini2026.03 | 44.58 | 36.66 | 75.66 | |
| A-MemBackbone=GPT-4o-mini2026.03 | 37.9 | 28.85 | 52.63 | |
| A-MemBackbone=Claude-Haiku-4.52026.03 | 36.12 | 27.1 | 52.63 | |
| Full TextBackbone=GPT-4o-mini2026.03 | 34.13 | 24.63 | 61.18 | |
| Full TextBackbone=Claude-Haiku-4.52026.03 | 34.13 | 24.63 | 61.18 | |
| Naive RAGBackbone=GPT-4o-mini2026.03 | 27.14 | 20.41 | 46.05 | |
| Naive RAGBackbone=Claude-Haiku-4.52026.03 | 27.14 | 20.41 | 46.05 | |
| LangMemBackbone=Claude-Haiku-4.52026.03 | 24.81 | 18.78 | 44.74 | |
| LangMemBackbone=GPT-4o-mini2026.03 | 18.46 | 14.05 | 30.26 |