Long-horizon memory Question Answering on LongMemEval
68.4AccuracyGPT-4o (ceiling)
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-4o (ceiling)Backbone=GPT-4o, Answer Ctx=120,000∗2026.05 | 68.4 | |
| GPT-4o (matched)Backbone=GPT-4o, Answer Ctx=9,800∗2026.05 | 67.8 | |
| MemFlowBackbone=Qwen3-1.7B, Answer Ctx=2,2232026.05 | 61.8 | |
| ReActBackbone=Qwen3-1.7B, Answer Ctx=8,7322026.05 | 50.2 | |
| RAGBackbone=Qwen3-1.7B, Answer Ctx=2,2662026.05 | 46 | |
| Direct QA (full)Backbone=Qwen3-1.7B, Answer Ctx=8,6142026.05 | 42.2 | |
| MemGPTBackbone=Qwen3-1.7B, Answer Ctx=∼1,260†2026.05 | 38.8 | |
| Direct QA (short)Backbone=Qwen3-1.7B, Answer Ctx=2,4582026.05 | 34.2 | |
| MemobaseBackbone=Qwen3-1.7B, Answer Ctx=∼260†2026.05 | 13.4 |