Single-document Question Answering on Singledoc QA
32.06DuReader ScoreMemSearch-o1
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MemSearch-o1Backbone=Qwen 2.5-72B-Instruct2026.04 | 32.06 | 17.3 | 36.18 | 49.05 | 59.91 | 45.53 | |
| Direct RAGBackbone=Qwen 2.5-72B-Instruct2026.04 | 27.49 | 17.73 | 35.94 | 45.43 | 52.29 | 38.51 | |
| MemSearch-o1Backbone=Deepseek V3.12026.04 | 27.23 | 23.04 | 37.94 | 52.26 | 56.81 | 48.17 | |
| AmberBackbone=Qwen 2.5-72B-Instruct2026.04 | 20.57 | 16.65 | 27.19 | 37.98 | 52.26 | 38.4 | |
| MemoryBankBackbone=Qwen 2.5-72B-Instruct2026.04 | 19.24 | 16.26 | 26.28 | 32.42 | 43.46 | 32.26 | |
| Search-o1 (RAgent)Backbone=Qwen 2.5-72B-Instruct2026.04 | 18.91 | 14.37 | 22.88 | 37.77 | 49.97 | 38.62 | |
| A-MemBackbone=Qwen 2.5-72B-Instruct2026.04 | 18.74 | 15.18 | 22.09 | 34.59 | 50.79 | 36.96 | |
| Direct RAGBackbone=Deepseek V3.12026.04 | 18.72 | 21.48 | 32.45 | 43.41 | 48.15 | 36.11 | |
| AmberBackbone=Deepseek V3.12026.04 | 18.39 | 22.48 | 32.86 | 39.27 | 49.31 | 39.79 | |
| MemoryBankBackbone=Deepseek V3.12026.04 | 17.57 | 21.83 | 28.6 | 33.34 | 42.33 | 34.49 | |
| A-MemBackbone=Deepseek V3.12026.04 | 17.1 | 20.92 | 27.55 | 28.27 | 47.29 | 32.22 | |
| Search-o1 (Refined)Backbone=Deepseek V3.12026.04 | 16.12 | 18.59 | 22.97 | 33.82 | 44.79 | 31.85 | |
| Search-o1 (Refined)Backbone=Qwen 2.5-72B-Instruct2026.04 | 15.3 | 16.12 | 17.51 | 33.79 | 44.45 | 31.63 | |
| Search-o1 (RAgent)Backbone=Deepseek V3.12026.04 | 15.28 | 20.59 | 33.04 | 38.67 | 49.76 | 39.05 |