Multi-document Question Answering on MultidocQA
67.78HotpotQA AccuracyMemSearch-o1
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MemSearch-o1Backbone=Deepseek V3.12026.04 | 67.78 | 68.32 | 52.01 | |
| MemSearch-o1Backbone=Qwen 2.5-72B-Instruct2026.04 | 59.71 | 65.95 | 44.11 | |
| Search-o1 (RAgent)Backbone=Qwen 2.5-72B-Instruct2026.04 | 57.41 | 62.86 | 44.78 | |
| AmberBackbone=Deepseek V3.12026.04 | 55.59 | 58.63 | 41.29 | |
| Search-o1 (RAgent)Backbone=Deepseek V3.12026.04 | 54.64 | 56.14 | 44.31 | |
| Direct RAGBackbone=Qwen 2.5-72B-Instruct2026.04 | 54.4 | 47.23 | 27.54 | |
| A-MemBackbone=Qwen 2.5-72B-Instruct2026.04 | 54.24 | 60.08 | 39.95 | |
| Direct RAGBackbone=Deepseek V3.12026.04 | 53.98 | 43.76 | 26.95 | |
| AmberBackbone=Qwen 2.5-72B-Instruct2026.04 | 53.78 | 61.16 | 37.61 | |
| Search-o1 (Refined)Backbone=Qwen 2.5-72B-Instruct2026.04 | 52.11 | 35.74 | 37.99 | |
| A-MemBackbone=Deepseek V3.12026.04 | 49.57 | 41.38 | 25.67 | |
| Search-o1 (Refined)Backbone=Deepseek V3.12026.04 | 48.46 | 36.75 | 33.32 | |
| MemoryBankBackbone=Deepseek V3.12026.04 | 47.51 | 46.65 | 38.05 | |
| MemoryBankBackbone=Qwen 2.5-72B-Instruct2026.04 | 47.27 | 44.89 | 28.26 |