Multi-hop Question Answering on HotpotQA (Context Length F1)
70.32F1 (56K Context)MemPro-15
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MemPro-15Backbone=GPT-4o-mini, Evolution Iterations=152026.05 | 70.32 | 65.81 | 64.02 | 66.72 | |
| MemPro-10Backbone=GPT-4o-mini, Evolution Iterations=102026.05 | 68.05 | 63.94 | 62.36 | 64.78 | |
| MemPro-15Backbone=Qwen2.5-14B, Evolution Iterations=152026.05 | 66.98 | 62.28 | 59.8 | 63.02 | |
| MemPro-10Backbone=Qwen2.5-14B, Evolution Iterations=102026.05 | 65.41 | 60.54 | 58.76 | 61.57 | |
| MemPro-5Backbone=GPT-4o-mini, Evolution Iterations=52026.05 | 65.18 | 61.72 | 60.84 | 62.58 | |
| MemPro-5Backbone=Qwen2.5-14B, Evolution Iterations=52026.05 | 64.36 | 58.73 | 57.91 | 60.33 | |
| GAMBackbone=Qwen2.5-14B2026.05 | 64.07 | 55.99 | 57.87 | 59.31 | |
| GAMBackbone=GPT-4o-mini2026.05 | 63.22 | 61.46 | 59.81 | 61.5 | |
| Full TextBackbone=GPT-4o-mini2026.05 | 56.56 | 54.29 | 53.92 | 54.92 | |
| RAGBackbone=GPT-4o-mini2026.05 | 52.71 | 51.84 | 54.01 | 52.85 | |
| RAGBackbone=Qwen2.5-14B2026.05 | 51.81 | 46.72 | 48.36 | 48.96 | |
| Full TextBackbone=Qwen2.5-14B2026.05 | 49.75 | 46.82 | 43.17 | 46.58 | |
| LightMemBackbone=GPT-4o-mini2026.05 | 40.93 | 35.28 | 30.02 | 35.41 | |
| LightMemBackbone=Qwen2.5-14B2026.05 | 37.3 | 27.72 | 28.25 | 31.09 | |
| A-MEMBackbone=GPT-4o-mini2026.05 | 33.9 | 30.22 | 31.37 | 31.83 | |
| Mem0Backbone=GPT-4o-mini2026.05 | 32.58 | 31.74 | 27.41 | 30.58 | |
| Mem0Backbone=Qwen2.5-14B2026.05 | 30.12 | 32.44 | 26.55 | 29.7 | |
| A-MEMBackbone=Qwen2.5-14B2026.05 | 27.04 | 25.65 | 22.92 | 25.2 | |
| MemoryOSBackbone=GPT-4o-mini2026.05 | 26.47 | 23.1 | 24.16 | 24.58 | |
| MemoryOSBackbone=Qwen2.5-14B2026.05 | 24.58 | 30.25 | 23.13 | 25.99 |