Long-Memory Question Answering on LoCoMo (Accuracy Metrics)
97.5Accuracy (Single-Hop)ByteRover
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| ByteRoverEvaluation Judge=Gemini 3 Flash2026.04 | 97.5 | 93.3 | 85.9 | 97.8 | 96.1 | — | |
| HyperMemevaluator=GPT-4o-mini2026.04 | 96.08 | 93.62 | 70.83 | 89.72 | 92.73 | — | |
| HonChoEvaluation Judge=Gemini 3 Flash2026.04 | 93.2 | 84 | 77.1 | 88.2 | 89.9 | — | |
| HyperGraphRAGevaluator=GPT-4o-mini2026.04 | 90.61 | 80.85 | 70.83 | 85.36 | 86.49 | — | |
| LightRAGevaluator=GPT-4o-mini2026.04 | 86.68 | 84.04 | 71.88 | 60.75 | 79.87 | — | |
| HippoRAGevaluator=GPT-4o-mini2026.04 | 86.44 | 75.89 | 66.67 | 78.5 | 81.62 | — | |
| HindsightEvaluation Judge=Gemini 3 Flash2026.04 | 86.2 | 70.8 | 95.1 | 83.8 | 89.6 | — | |
| MIRIXevaluator=GPT-4.1-mini2026.04 | 85.11 | 83.7 | 65.62 | 88.39 | 85.38 | — | |
| MemOSevaluator=GPT-4o-mini2026.04 | 81.09 | 67.49 | 55.9 | 75.18 | 75.8 | — | |
| GraphRAGevaluator=GPT-4o-mini2026.04 | 79.55 | 54.96 | 58.33 | 50.16 | 67.6 | — | |
| ZepEvaluation Judge=Gemini 3 Flash2026.04 | 74.1 | 66 | 67.7 | 79.8 | 75.1 | — | |
| Memobaseevaluator=GPT-4o-mini, index=42026.04 | 73.12 | 64.65 | 53.12 | 81.2 | 72.01 | — | |
| MemobaseEvaluation Judge=Gemini 3 Flash2026.04 | 70.9 | 46.9 | 77.2 | 85.1 | 75.8 | — | |
| Mem0evaluator=GPT-4o-mini2026.04 | 67.13 | 51.15 | 72.93 | 55.51 | 66.88 | — | |
| Mem0Evaluation Judge=Gemini 3 Flash2026.04 | 67.1 | 51.2 | 72.9 | 55.5 | 66.9 | — | |
| MemUevaluator=GPT-4o-mini, index=52026.04 | 66.34 | 63.12 | 50.01 | 27.1 | 56.55 | — | |
| Mem0^gevaluator=GPT-4o-mini2026.04 | 65.71 | 47.19 | 75.71 | 58.13 | 68.44 | — | |
| OpenAI MemoryEvaluation Judge=Gemini 3 Flash2026.04 | 63.8 | 42.9 | 62.3 | 21.7 | 52.9 | — | |
| OpenAIevaluator=GPT-4o-mini, index=22026.04 | 63.79 | 42.92 | 62.29 | 21.71 | 52.9 | — | |
| LangMemevaluator=GPT-4o-mini, index=32026.04 | 62.23 | 47.92 | 71.12 | 23.43 | 58.1 | — | |
| Zepevaluator=GPT-4o-mini2026.04 | 61.7 | 41.35 | 76.6 | 49.31 | 65.99 | — | |
| A-Memevaluator=GPT-4o-mini2026.04 | 39.79 | 18.85 | 54.05 | 49.91 | 48.38 | — | |
| Direct QA (full)Backbone=Qwen3-1.7B, Answer Ctx=8,6142026.05 | — | — | — | — | — | 33.7 | |
| Direct QA (short)Backbone=Qwen3-1.7B, Answer Ctx=2,4582026.05 | — | — | — | — | — | 24 | |
| GPT-4o (ceiling)Backbone=GPT-4o, Answer Ctx=120,000∗2026.05 | — | — | — | — | — | 76.3 | |
| GPT-4o (matched)Backbone=GPT-4o, Answer Ctx=9,800∗2026.05 | — | — | — | — | — | 56.6 | |
| MemFlowBackbone=Qwen3-1.7B, Answer Ctx=2,2232026.05 | — | — | — | — | — | 51.2 | |
| MemGPTBackbone=Qwen3-1.7B, Answer Ctx=∼1,260†2026.05 | — | — | — | — | — | 31.6 | |
| MemobaseBackbone=Qwen3-1.7B, Answer Ctx=∼260†2026.05 | — | — | — | — | — | 7.7 | |
| RAGBackbone=Qwen3-1.7B, Answer Ctx=2,2662026.05 | — | — | — | — | — | 45.9 | |
| ReActBackbone=Qwen3-1.7B, Answer Ctx=8,7322026.05 | — | — | — | — | — | 49.6 |